Information processing device, information processing method, information processing program, and microscope for displaying a plurality of surface images

ABSTRACT

An information processing device comprises: an image processor which generates surface information from point cloud data generated on the basis of position information of an object, using a value of a first parameter and a value of a second parameter; and a display controller which causes a display to display a surface image on the basis of the generated surface information, wherein the image processor generates a plurality of pieces of surface information, using a plurality of values of the first parameter and a plurality of values of the second parameter, and the display controller causes the display to display a plurality of surface images on the basis of the plurality of pieces of generated surface information.

CROSS REFERENCE TO RELATED APPLICATION

This is a Continuation of PCT Application No. PCT/JP2018/020865, filedon May 30, 2018. The contents of the above-mentioned application areincorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to an information processing device, aninformation processing method, an information processing program, and amicroscope.

BACKGROUND

STORMS, PALMs, and the like, for example, are known as super-resolutionmicroscopes. In STORMS, a fluorescent substance is activated and theactivated fluorescent substance is irradiated with excitation light tothereby acquire a fluorescent image (see Patent Literature 1 below).

CITATION LIST Patent Literature

[Patent Literature 1] US Patent Application Publication No. 2008/0182336

SUMMARY

A first aspect of the invention is to provide an information processingdevice comprising: an image processor which generates surfaceinformation from point cloud data generated on the basis of positioninformation of an object, using a value of a first parameter and a valueof a second parameter; and a display controller which causes a displayto display a surface image on the basis of the generated surfaceinformation, wherein the image processor generates a plurality of piecesof surface information, using a plurality of values of the firstparameter and a plurality of values of the second parameter, and thedisplay controller causes the display to display a plurality of surfaceimages on the basis of the plurality of pieces of generated surfaceinformation.

The display controller may be configured to cause the display to displaya plurality of surface images in an arranged manner.

The display controller may be configured to cause the display to displaya surface image at a position corresponding to the value of eachparameter where the vertical axis represents the values of the firstparameter and the horizontal axis represents the values of the secondparameter. Alternatively, the display controller may be configured tocause the display to display, in a manner of being arranged in a rowdirection, a plurality of surface images corresponding to a plurality ofpieces of surface information generated using a plurality of values ofthe first parameter, and display, in a manner of being arranged in acolumn direction, a plurality of surface images corresponding to aplurality of pieces of surface information generated using a pluralityof values of the second parameter. Alternatively, the display controllermay be configured to cause the display to display, in a manner of beingarranged in a vertical axis direction, a plurality of surface imagescorresponding to a plurality of pieces of surface information generatedusing a plurality of values of the first parameter, and display, in amanner of being arranged in a horizontal axis direction, a plurality ofsurface images corresponding to a plurality of pieces of surfaceinformation generated using a plurality of values of the secondparameter.

Any of the above display controllers may be configured to cause thedisplay to display a setting screen to set the value of a firstparameter and the value of a second parameter, and display, on thesetting screen, the value of a first parameter and the value of a secondparameter corresponding to a surface image selected from a plurality ofsurface images displayed on the display.

The image processor may be configured such that it calculates a scalarfield, applying a predetermined function in a region within apredetermined distance from each point data of the point cloud data,discretizes a scalar field at a grid point, and calculates the surfaceinformation, calculating an isosurface of a scalar value of the gridpoint, and that the value of the first parameter and the value of thesecond parameter are any one of a value related to the size of theregion, a value related to the resolution of the grid point, and a valueof the isosurface.

A second aspect of the invention is to provide a microscope comprising:the information processing device of the first aspect; an illuminationoptical system which irradiates an excitation light to excite afluorescent substance contained in a sample; an observation opticalsystem which forms an image of light from the sample; and an imagecapturer which captures the image formed by the observation opticalsystem, wherein the image processor calculates position information ofthe fluorescent substance on the basis of a result captured by the imagecapturer, and generates the point cloud data, using the calculatedposition information.

A third aspect of the invention is to provide a microscope comprising:the information processing device of the first aspect; an optical systemwhich irradiates activation light for activating a part of a fluorescentsubstance contained in a sample; an illumination optical system whichirradiates an excitation light to excite at least a part of theactivated fluorescent substance; an observation optical system whichforms an image of light from the sample; and an image capturer whichcaptures the image formed by the observation optical system, wherein theimage processor calculates position information of the fluorescentsubstance on the basis of a result captured by the image capturer, andgenerates the point cloud data, using the calculated positioninformation.

A fourth aspect of the invention is to provide an information processingmethod for generating surface information from point cloud datagenerated on the basis of position information of an object, using avalue of a first parameter and a value of a second parameter, andcausing a display to display a surface image on the basis of thegenerated surface information, the information processing methodcomprising: generating a plurality of pieces of surface information,using a plurality of values of the first parameter and a plurality ofvalues of the second parameter; and causing the display to display aplurality of surface images on the basis of the plurality of pieces ofgenerated surface information.

A fifth aspect of the invention is to provide a computer-readablenon-transitory tangible medium containing an information processingprogram which causes a computer to execute processes of generatingsurface information from point cloud data generated on the basis ofposition information of an object, using a value of a first parameterand a value of a second parameter, and causing a display to display asurface image on the basis of the generated surface information, theinformation processing program including processes of: generating aplurality of pieces of surface information, using a plurality of valuesof the first parameter and a plurality of values of the secondparameter; and causing the display to display a plurality of surfaceimages on the basis of the plurality of pieces of generated surfaceinformation.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing an information processing device accordingto a first embodiment.

FIG. 2A to FIG. 2C are diagrams showing an example of point cloud dataand a function of a local field according to the first embodiment.

FIG. 3A to FIG. 3D are diagrams showing a process of generating afunction which represents a scalar field according to the firstembodiment.

FIG. 4A to FIG. 4B are diagrams showing an example different from thatof FIG. 2A to FIG. 2C, concerning the function of a local fieldaccording to the first embodiment.

FIG. 5 is a diagram showing a process of discretizing the scalar fieldin the first embodiment.

FIG. 6A to FIG. 6E are diagrams showing a process of generating anisosurface in the first embodiment.

FIG. 7 is a diagram showing a process of generating an isosurface in thefirst embodiment.

FIG. 8A to FIG. 8C are diagrams showing isosurfaces on the basis ofscalar values in the first embodiment.

FIG. 9A to FIG. 9D are diagrams showing isosurfaces on the basis ofresolutions in the first embodiment.

FIG. 10A and FIG. 10B are diagrams showing isosurfaces on the basis ofradiuses in the first embodiment.

FIG. 11 is a diagram showing a process of a UI according to the firstembodiment.

FIG. 12 is a diagram showing a setting process using the UI according tothe first embodiment.

FIG. 13 is a diagram showing a setting process using the UI according tothe first embodiment.

FIG. 14 is a diagram showing images displayed by the UI according to thefirst embodiment.

FIG. 15 is a diagram showing images displayed by the UI according to thefirst embodiment.

FIG. 16 is a diagram showing images displayed by the UI according to thefirst embodiment.

FIG. 17 is a flowchart showing an information processing methodaccording to the first embodiment.

FIG. 18 is a diagram showing an information processing device accordingto a second embodiment.

FIG. 19 is a diagram showing a process of a UI according to the secondembodiment.

FIG. 20 is a diagram showing a process of the UI according to the secondembodiment.

FIG. 21 is a diagram showing a process of the UI according to the secondembodiment.

FIG. 22 is a diagram showing a process of the UI according to the secondembodiment.

FIG. 23 is a diagram showing a process of the UI according to the secondembodiment.

FIG. 24 is a diagram showing a process of the UI according to the secondembodiment.

FIG. 25 is a diagram showing a microscope according to the embodiment.

FIG. 26 is a diagram showing a microscope main body according to theembodiment.

DETAILED DESCRIPTION OF EMBODIMENTS First Embodiment

Hereunder, a first embodiment will be described. FIG. 1 is a diagramshowing an information processing device according to a firstembodiment. An information processing device 1 according to theembodiment generates an image (a point cloud image), using point clouddata DG, and displays it on a display device 2. The informationprocessing device 1 processes the point cloud data DG. The point clouddata DG is a set of a plurality of pieces of N-dimensional data D1(where N is an any integer of 2 or greater). The N-dimensional data D1is a piece of data (for example, vector data) in which N values aregrouped as a set. For example, in FIG. 1, the point cloud data DG is apiece of three-dimensional data in which coordinate values in athree-dimensional space (for example, x1, y1, z1) are grouped as a set.In the following description, it is assumed that N mentioned above is 3.N may be 2, or 4 or greater. In FIG. 1, the point cloud data DG is mpieces of N-dimensional data. m is any integer 2 or greater.

The point cloud image is an image generated using the point cloud dataDG. For example, in the case where the point cloud data DG is a piece ofthree-dimensional data in which coordinate values in a three-dimensionalspace are grouped as a set (for example, x1, y1, z1), the point cloudimage is an image displaying a point at each coordinate position. Thesize of the displayed point can be changed as appropriate. The shape ofthe displayed point is not limited to a circular shape, and may beanother shape such as an elliptical shape and a rectangular shape. Pointcloud data is sometimes referred to simply as point cloud. In thepresent specification, a plurality of points on a point cloud image arereferred to as point cloud where appropriate.

The point cloud data DG is supplied, for example, from an externaldevice of the information processing device 1 (hereunder, referred to asexternal device) to the information processing device 1. Theabove-mentioned external device is, for example, a microscope main body51 shown later in FIG. 25. The above-mentioned external device need notbe the microscope main body 51. For example, the external device may bea CT scanner to detect the value at each point within an object, or ameasuring device to measure the shape of an object. The informationprocessing device 1 may generate point cloud data DG on the basis ofdata supplied from the external device and may process the generatedpoint cloud data DG.

The information processing device 1 executes processing on the basis ofinput information input by the user using a graphical user interface.The information processing device 1 is connected to the display device 2(the display). The display device 2 is, for example, a liquid crystaldisplay or the like. The information processing device 1 supplies dataof an image to the display device 2 and causes the display device 2 todisplay the image. The display device 2 is, for example, a deviceexternally attached to the information processing device 1. However, thedisplay device 2 may be a part of the information processing device 1.

The information processing device 1 is connected to an input device 3(an inputter). The input device 3 is an input interface which can beoperated by the user. The input device 3 includes, for example, at leastone of a mouse, a keyboard, a touch pad, and a trackball. The inputdevice 3 detects an operation performed by the user and supplies thedetection result to the information processing device 1 as inputinformation input by the user.

In the following description, it is assumed that the input device 3 is amouse. When the input device 3 is a mouse, the information processingdevice 1 causes the display device 2 to display a pointer. Theinformation processing device 1 acquires mouse movement information andclick information from the input device 3 as input information detectedby the input device 3. The mouse movement information represents, forexample, an amount of movement the mouse makes. The click informationrepresents whether or not a button of the mouse has been operated (forexample, whether or not a click has been performed).

The information processing device 1 causes the pointer to move on thescreen of the display device 2 on the basis of the mouse movementinformation. The information processing device 1 executes, on the basisof click information, a process assigned to the position of the pointerand the click information (for example, a left-click, a right-click, adrag, or a double-click). The input device 3 is, for example, a deviceexternally attached to the information processing device 1, however, itmay be a part of the information processing device 1 (for example, abuilt-in touch pad). The input device 3 may be a touch panel or the likeintegrated with the display device 2.

The information processing device 1 includes, for example, a computer.The information processing device 1 includes an operating system 5(hereunder, referred to as OS 5), a UI 6 (a user interface), an imageprocessor 7, and a memory storage 9. The information processing device 1executes various processes according to a program stored in the memorystorage 9. The OS 5 provides an interface between the outside and theinside of the information processing device 1. For example, the OS 5controls the supply of image data to the display device 2. The OS 5acquires input information from the input device 3. The OS 5 suppliesuser input information to, for example, an application which manages anactive GUI screen on the display device 2. For example, when a windowmanaged by the application executed by the process of the UI 6 isactive, user input information is provided to the UI 6.

The UI 6 includes an input controller 11 and an output controller 12.The output controller 12 is a display controller which causes thedisplay (the display device 2) to display a setting screen to setcalculation conditions (such as setting information, setting values,parameter values, surface information generation conditions) used togenerate surface information. The setting screen is, for example, a GUIscreen W shown later in FIG. 11 to FIG. 16 and so forth. The GUI screenW is a window or the like provided by an application. The displaycontroller (for example, the output controller 12) displays, on thedisplay (the display device 2), an image representing surfaceinformation generated by the image processor 7. The display controller(for example, the output controller 12) may display the setting screenand the image representing surface information in the same region (forexample, the same window) on the display device 2. The displaycontroller (for example, the output controller 12) may display thesetting screen and the image representing surface information indifferent regions (for example, a first window and a second window) onthe display device 2.

Information constituting the GUI screen W (hereunder, referred to as GUIinformation) is stored in, for example, the memory storage 9. The outputcontroller 12 reads out GUI information from the memory storage 9 andsupplies the GUI information to the OS 5. The OS 5 causes the displaydevice 2 to display the GUI screen on the basis of the GUI informationsupplied from the output controller 12. Thus, the output controller 12causes the display device 2 to display the GUI screen by supplying GUIinformation to the OS 5.

The input controller 11 acquires input information input by the userusing the GUI screen. For example, the input controller 11 acquiresmouse movement information and click information as input informationfrom the OS 5. When the click information indicates a click operationhaving been performed, the input controller 11 executes a processassigned to the click operation indicated by the click information, onthe basis of the coordinates of the pointer on the GUI screen obtainedfrom the mouse movement information.

For example, it is assumed that a right-click has been detected ashaving been performed on the GUI screen. It is also assumed that theprocess assigned to a right-click is a process of displaying a menu. Insuch a case, the input controller 11 causes the output controller 12 toexecute the process of displaying the menu. Information which representsthe menu is included in the GUI information, and the output controller12 causes the display device 2 to display the menu via the OS 5 on thebasis of the GUI information.

Now, it is assumed that a left-click has been detected as having beenperformed on the GUI screen. It is preliminarily defined that if aleft-click is preformed while the pointer is being placed over a buttonon the GUI screen, the process assigned to this button is executed. Theinput controller 11 identifies the position of the pointer on the GUIscreen on the basis of mouse movement information, and determineswhether or not a button is present at the identified position of thepointer. If a left-click is detected as having been performed in thestate where a button is present at the position of the pointer, theinput controller 11 executes the process assigned to this button.

The image processor 7 generates surface information on the basis of thepoint cloud data DG, using calculation conditions set on the settingscreen (for example, the GUI screen W). The image processor 7 includes afield calculator 7A and a surface generator 7B. The field calculator 7Acalculates a scalar field on the basis of the point cloud data DG. Thefield calculator 7A calculates a function representing a scalar field onthe basis of the point cloud data DG including the coordinates of aplurality of points.

The field calculator 7A generates a local scalar field on the basis ofeach of one or more pieces of point data of (for each of one or morepieces of point data of) the point cloud data DG, and calculates ascalar field on the basis of the local scalar field. The fieldcalculator 7A calculates (generates) a function representing a scalarfield on the basis of a local field in the case where the source originof the local field is arranged at each of one or more point among theplurality of points included in the point cloud data DG. A process ofcalculating a function representing a scalar field (referred to as fieldcalculation process where appropriate) will be described, with referenceto FIG. 2A to FIG. 2C, FIG. 3A to FIG. 3D, FIG. 4A to FIG. 4B, and FIG.5. In FIG. 2A to FIG. 2C, FIG. 3A to FIG. 3D, FIG. 4A to FIG. 4B, andFIG. 5, processes are described on a two-dimensional plane, however,extending the processes to three dimensions would enable processing ofthree-dimensional point cloud data DG in a similar manner.

FIG. 2A to FIG. 2C are diagrams showing an example of the point clouddata and a function of a local field according to the first embodiment.In FIG. 2A, reference signs P1 to P3 each denote points included in thepoint cloud data DG. Here, the coordinates of the point P1 arerepresented by (x1, y1), the coordinates of the point P2 are representedby (x2, y2), and the coordinates of the point P3 are represented by (x3,y3).

FIG. 2B and FIG. 2C are diagrams showing an example of a local field (alocal scalar field). Reference sign LF in FIG. 2B denotes a local field,and reference sign FG denotes the source origin of the local field LF.The local field (a local scalar field) is a function in which the valuechanges stepwise with respect to the distance from each point datawithin a range of a predetermined distance from each point data. Forexample, a function which represents a local field is a function inwhich the value (the scalar value) changes stepwise with respect to thedistance from the argument point to a predetermined point (for example,a source origin FG). As shown in FIG. 2C, the function f1 representingthe local field LF is a function in which the value changes stepwisewith respect to a distance r from the source origin FG (the origin 0).The value of f1 at each point of field (argument point) is defined bythe distance r from a predetermined point (a source origin)corresponding to each point data selected from the point cloud data toeach point of field, and the value within a region where the distance ris 0 or more and less than r1 (for example, 1) and the value within aregion where the distance r is r1 or more (for example, 0) changesstepwise.

The local scalar field (the local field) may be a function in which thevalue is constant in a region where the distance from each point data isequal to or greater than a threshold value. The function representingthe local field may be a function in which the value (the scalar value)is constant in a region where the distance from the argument point to apredetermined point (for example, the source origin FG) is equal to orgreater than a threshold value. For example, the function f1 in FIG. 2Cis a function in which the value is constant (for example, 0) in theregion where the distance r is equal to or greater than a thresholdvalue (r1).

The local scalar field (the local field) may be a function where thevalue decreases with respect to the distance from each point data withina range of a predetermined distance from each point data. The functionrepresenting the local field may be a function in which the value (thescalar value) decreases with respect to the distance from the argumentpoint to a predetermined point (for example, the source origin FG). Thevalue of the function f1 in FIG. 2C is a function in which the value inthe region where the distance r is r1 or more (for example, 0) decreaseswith respect to the value in the region where the distance r is 0 ormore and less than r1 (for example 1). The function f1 of FIG. 2C is afunction in which the value decreases with respect to the distance rfrom the source origin FG, and is a function which represents anattenuation field as the local field LF.

FIG. 3A to FIG. 3D are diagrams showing a process of generating afunction representing a scalar field according to the first embodiment.The upper figure of FIG. 3A shows a local field LF1 originating from thepoint P1. The lower figure of FIG. 3A shows a distribution of the valuev of the local field LF1 where y=y1 (indicated by the dotted line in theupper figure of FIG. 3A). The upper figure of FIG. 3B shows a localfield LF2 originating from the point P2. The lower figure of FIG. 3Bshows a distribution of the value v of the local field LF2 where y=y1(indicated by the dotted line in the upper figure of FIG. 3B). The upperfigure of FIG. 3C shows a local field LF3 originating from the point P1.The lower figure of FIG. 3C shows a distribution of the value v of thelocal field LF3 where y=y1 (indicated by the dotted line in the upperfigure of FIG. 3C).

The upper figure of FIG. 3D shows a scalar field FS formed by the localfield LF1 to the local field LF3. The scalar field FS is represented bysuperposing the local field LF1, the local field LF2, and the localfield LF3. The lower figure of FIG. 3D shows a distribution of the valuev of the scalar field FS where y=y1 (indicated by the dotted line in theupper figure of FIG. 3D). The value v of the scalar field FS is 0 in aregion where none of the local field LF1 to the local field LF3 ispresent. The value v of the scalar field FS is 1 in a region where onlyone local field among the local field LF1 to the local field LF3 ispresent. The value v of the scalar field FS is 2 in a region where twoof the local fields among the local field LF1 to the local field LF3 aresuperposed. The value v of the scalar field FS is 3 in a region wherethe local field LF1, the local field LF2, and the local field LF3 areall superposed.

The field calculator 7A (see FIG. 1) calculates the distribution of thevalue v in the scalar field FS by superimposing the local fields (LF1,LF2, LF3) originating from the respective points (P1, P2, P3). Forexample, the field calculator 7A applies a filter (for example,Laplacian, or Gaussian) to the function that represents thesuperposition of the local fields to calculate a function thatrepresents a scalar field. The function representing the scalar fieldmay be represented by a mathematical expression, or may be representedby table data which combines coordinate values and a scalar value as aset. As a function representing the scalar field FS, the fieldcalculator 7A may calculate a coefficient of a mathematical expressionin a preliminarily set function form, or may calculate the table datamentioned above.

The form of local scalar fields (local fields) is not limited to theform of FIG. 2B. FIG. 4A to FIG. 4B are diagrams showing an exampledifferent from that of FIG. 2A, FIG. 2B, and FIG. 2C, concerning thefunction of the local field according to the first embodiment. The localscalar field (the local field) may be a function where the valuecontinuously changes with respect to the distance from each point datawithin a range of a predetermined distance from each point data. Thefunction representing the local field may be a function in which thevalue (the scalar value) continuously changes with respect to thedistance from the argument point thereof to a predetermined point (forexample, the source origin FG).

In FIG. 4A, the function f4 representing the local field LF4 is afunction in which within a predetermined range (for example, a rangewhere r is 0 or more) of the distance from the position (the sourceorigin FG, the predetermined position) of each point data selected fromthe point cloud data, the value v continuously changes with respect tothe distance r from the position (the source origin FG, thepredetermined position) of each point data to each point of the field.The function f4 is a function in which the value decreases with respectto the distance from the source origin FG. The function f4 is, forexample, a function representing a Gaussian distribution, however, itmay be another function. In FIG. 4B, the function f5 is a function inwhich within a predetermined range (for example, a range where r is 0 ormore and less than r2) of the distance from the position (the sourceorigin FG, the predetermined position) of each point data selected fromthe point cloud data, the value v continuously changes with respect tothe distance r from the position (the source origin FG, thepredetermined position) of each point data to each point of the field.The function f5 is a function in which the value is constant (forexample, 0) in a region where the distance r from the source origin FGis equal to or greater than the threshold value r2.

The surface generator 7B shown in FIG. 1 generates surface informationrepresenting an isosurface in the scalar field calculated by the fieldcalculator 7A. The isosurface mentioned above is a surface in which thescalar value is a user-specified value in the scalar field. The inputcontroller 11 is an input information acquirer which acquires a scalarvalue input by the inputter (the input device 3). The surface generator7B calculates the isosurface of a scalar field on the basis of thescalar value acquired by the input information acquirer (for example,the input controller 11) and generates surface information. The processof acquiring a specified value of scalar value will be described laterwith reference to FIG. 12 and so forth.

The surface generator 7B discretizes a scalar field at a grid point andgenerates surface information using the scalar value of the grid point.The surface generator 7B discretizes the scalar field into a value (ascalar value) at the grid point. The surface generator 7B generatessurface information using data in which the function representing thescalar field is represented as a scalar value on the grid point. Thesurface generator 7B discretizes the scalar field to generate surfaceinformation.

FIG. 5 is a diagram showing the process of discretizing a scalar fieldin the first embodiment. In FIG. 5, reference sign GLx denotes gridlines parallel to the x axis, and reference sign dy denotes intervalsbetween the grid lines GLx. Reference sign GLy denotes grid linesparallel to the y axis, and reference sign dx denotes intervals betweenthe grid lines GLy. The grid points P4 are intersections between thegrid lines GLx and the grid lines GLy. The interval between adjacentgrid points P4 on the grid lines GLx is an interval dx of the grid linesGLy. The interval between adjacent grid points P4 on the grid lines GLyis an interval dy of the grid lines GLx. Here, the interval dx and theinterval dy are equal. In the following description, the interval dx andthe interval dy are collectively referred to as resolution d whereappropriate.

The surface generator 7B calculates the value of the scalar field FS ateach of the grid points P4. For example, among the grid points P4, thegrid point P4 a is a region where none of the local field LF1 to thelocal field LF3 is present, and the scalar value (v) is 0. Among thegrid points P4, the grid point P4 b is a region where only the localfield LF1 is present, and the scalar value (v) is 1. Among the gridpoints P4, the grid point P4 c is a region where the local field LF1 andthe local field LF2 is present, and the scalar value (v) is 2. Among thegrid points P4, the grid point P4 d is a region where all of the localfield LF1 to the local field LF3 are present, and the scalar value (v)is 3. The field calculator 7A may execute the process of discretizingthe scalar field. For example, as a function representing the scalarfield FS, the field calculator 7A may calculate table data whichcombines coordinate values and a scalar value as a set.

Next, with reference to FIG. 6A to FIG. 6E, FIG. 7, FIG. 8A to FIG. 8C,FIG. 9A to FIG. 9D, FIG. 10A, and FIG. 10B, the process of generating anisosurface (a constant-level surface) will be described. In FIG. 6A toFIG. 6E, FIG. 7, FIG. 8A to FIG. 8C, FIG. 9A to FIG. 9D, FIG. 10A, andFIG. 10B, the process is described on a two-dimensional plane, however,extending this process to three dimensions would enable processing ofthree-dimensional point cloud data DG in a similar manner. In FIG. 6A toFIG. 6E, FIG. 7, FIG. 8A to FIG. 8C, FIG. 9A to FIG. 9D, FIG. 10A, andFIG. 10B, isosurfaces are represented by isolines.

FIG. 6A to FIG. 6E are diagrams showing the process of generating anisosurface in the first embodiment. The surface generator 7B generatessurface information by means of the marching cube method or a method towhich the marching cube method is applied, for example. The surfacegenerator 7B generates polygon data representing an isosurface assurface information.

In FIG. 6A to FIG. 6E, reference signs P4 e to P4 h are respectively thegrid points P4 shown in FIG. 5. The surface generator 7B generatessurface information on the basis of the relationship between the valueof a scalar field at a first grid point (for example, the grid point P4e) and the value of a scalar field at a second grid point (for example,the grid point P4 f) adjacent to the first grid point. In FIG. 6A toFIG. 6E, the number (0 or 1) shown near each grid point is the scalarvalue of each grid point. Here, the scalar value at each grid point is 0or 1. The scalar values of the grid point P4 e, the grid point P4 f, thegrid point P4 g, and the grid point P4 h are represented as a set. Forexample, the scalar value set of FIG. 6A is (0, 0, 0, 0). A squareregion with the grid point P4 e, the grid point P4 f, the grid point P4g, and the grid point P4 h serving as vertices thereof is referred to asgrid G.

When the scalar value of the grid point P4 e is greater than a specifiedvalue and the scalar value of the grid point P4 f adjacent to the gridpoint P4 e is less than the specified value, the surface generator 7Bdetermines that an end of the isoline in the grid G exists between thegrid point P4 e and the grid point P4 f.

For example, the scalar value set of FIG. 6A is (0, 0, 0, 0), and thescalar values of adjacent grid points are all less than the specifiedvalue. In such a case, the surface generator 7B determines that anisoline is not present in the grid G.

The scalar value set of FIG. 6B is (1, 1, 0, 0), and the surfacegenerator 7B determines that an end of an isoline is present bothbetween the grid point P4 e and the grid point P4 h, and between thegrid point P4 f and the grid point P4 g. The surface generator 7B takesthe line connecting the midpoint P5 a between the grid point P4 e andthe grid point P4 h to the midpoint P5 b between the grid point P4 f andthe grid point P4 g as an isoline EL1.

The scalar value set of FIG. 6C is (1, 0, 1, 0). In such a case, an endof an isoline is present between the grid point P4 e and the grid pointP4 h, between the grid point P4 e and the grid point P4 f, between thegrid point P4 f and the grid point P4 g, and between the grid point P4 gand the grid point P4 h. In such a case, for example, two isolines arepresent in the grid G. The surface generator 7B derives isolines so thatthe isolines do not intersect each other, for example. For example, thesurface generator 7B takes the line connecting the midpoint P5 c betweenthe grid point P4 e and the grid point P4 f to the midpoint P5 d betweenthe grid point P4 e and the grid point P4 h as an isoline EL2. Thesurface generator 7B takes the line connecting the midpoint P5 e betweenthe grid point P4 f and the grid point P4 g to the midpoint P5 f betweenthe grid point P4 g and the grid point P4 h as an isoline EL3.

The scalar value set of FIG. 6D is (1, 1, 1, 0). The surface generator7B takes the line connecting the midpoint P5 g between the grid point P4e and the grid point P4 h to the midpoint P5 h between the grid point P4g and the grid point P4 h as an isoline EL4. The scalar value set ofFIG. 6E is (1, 1, 1, 1), and the scalar values of adjacent grid pointsare all greater than the specified value. In such a case, the surfacegenerator 7B determines that an isoline is not present in the grid G.

The surface generator 7B searches for an isoline in one grid G, and thensearches for an isoline in the adjacent grid G in a similar manner. Thesurface generator 7B generates surface information by connecting anisoline in a grid G to an isoline in the adjacent grid G. The surfacegenerator 7B may generate surface information while simplifying theshape represented by polygon data. For example, the surface generator 7Bmay apply a low-pass filter to connected isolines to generate surfaceinformation.

The process of generating surface information is not limited to theexample shown in FIG. 6A to FIG. 6E. FIG. 7 is a diagram showing aprocess of generating an isosurface in the first embodiment. In FIG. 6Ato FIG. 6E, the end of the isoline is set at the midpoint between thegrid points, however, in FIG. 7, the position of the end of an isolineis set by weighting, on the basis of the scalar value and the specifiedvalue of the grid point.

In FIG. 7, it is assumed that the scalar value set is (2, 1, 0, 0), andthe specified value of the scalar value is a value greater than 1 andless than 2. It is also assumed that the distance between the grid pointP4 e and an end P5 i of the isoline EL5 is L1, and the distance betweenthe grid point P4 f and the end P5 i of the isoline EL5 is L2. In thesurface generator 7B sets the position of the end P5 i so that the ratioof the distance L1 to the distance L2 equals to the ratio of thedifference between the scalar value (2) of the grid point P4 e and thespecified value (a) to the difference between the scalar value (1) ofthe grid point P4 f and the specified value (a).

For example, the surface generator 7B sets the distance L1 to(2−a)/(2−1). For example, when a=1.6, the distance L1 is 0.4 and thedistance L2 is 0.6. Thus, between the grid point P4 e and the grid pointP4 f, the end P5 i is set at a position closer to the grid point P4 e(the scalar value is 2), the scalar value of which is closer to thespecified value (a=1.6). Also between the grid point P4 e and the gridpoint P4 h, the surface generator 7B sets the distance L3 between thegrid point P4 e and the end P5 j of the isoline EL5 to (2−a)/(2−0) in asimilar manner. The surface generator 7B sets the distance L4 betweenthe grid point P4 h and the end P5 j of the isoline EL5 to (a−2)/(2−0).When a=1.6, the distance L3 is 0.2 and the distance L4 is 0.8. Thus,between the grid point P4 e and the grid point P4 h, the end P5 j is setat a position closer to the grid point P4 e (the scalar value is 2), thescalar value of which is closer to the specified value (a=1.6).

FIG. 8A to FIG. 8C are diagrams showing isosurfaces (isolines) on thebasis of scalar values in the first embodiment. Here, for the scalarfield of FIG. 5, an isosurface in the case where the specified value isodiffers is shown. FIG. 8A shows an isosurface EL6 in the case where thespecified value iso is v1. v1 is, for example, a value greater than 0and less than 1 (for example, 0.5). FIG. 8B shows an isosurface EL7 inthe case where the specified value iso is v2. v2 is, for example, avalue greater than 1 and less than 2 (for example, 1.5). FIG. 8C showsan isosurface EL8 in the case where the specified value iso is v3. v3is, for example, a value greater than 2 and less than 3 (for example,2.5).

Thus, the extracted isosurface also differs when the specified valuediffers. In the case where the scalar field is an attenuation field, ifthe specified value of the scalar value is set to a large value (forexample, FIG. 8C), the extracted isosurface will be smaller compared tothat in the case where the specified value of the scalar value is set toa small value (for example, FIG. 8A).

FIG. 9A to FIG. 9D are diagrams showing isosurfaces on the basis ofresolutions in the first embodiment. FIG. 9A shows apart of the scalarfield FS. FIG. 9B shows an isosurface EL9 in the case where theresolution d is set to d1 for the scalar field FS. FIG. 9C shows anisosurface EL10 in the case where the resolution d is set to d2 for thescalar field FS. Here, d2=(d1)/2. FIG. 9D shows an isosurface EL11 inthe case where the resolution d is set to d3 for the scalar field FS.Here, d3=(d1)/4. When the resolution d is set low as in FIG. 9D, theisosurface can be extracted more finely compared to the case where theresolution d is set high as in FIG. 9B. When the resolution d is sethigh as in FIG. 9B, the processing load becomes lower than that in thecase where the resolution d is set low as in FIG. 9D. Thus, theextracted isosurface also differs when the resolution differs.

FIG. 10A and FIG. 10B are diagrams showing isosurfaces on the basis ofradiuses of local fields in the first embodiment. FIG. 10A shows anisosurface EL12 in the case where the radius r of local fields is r1 andthe specified value of the scalar value is greater than 1 and less than2, in contrast to the scalar field FS using the local field LF shown inFIG. 2B (see FIG. 3D). FIG. 10B shows an isosurface EL13 when the radiusr of the local fields LF is r3. In FIG. 10A and FIG. 10B, r3 is set to avalue smaller than r1. In the case where the scalar field is anattenuation field, when the radius r is set small as in FIG. 10B, theextracted isosurface becomes smaller than that in the case where theradius r is set large as in FIG. 10A. Thus, the extracted isosurfacealso differs when the radiuse r differs.

As explained in FIG. 8A to FIG. 8C, FIG. 9A to FIG. 9D, FIG. 10A, andFIG. 10B, the isosurface extracted by the surface generator 7B differs,depending on conditions for calculating the isosurface (for example, thespecified value iso of the scalar value, the resolution d, the radiusr). The UI 6 shown in FIG. 1 acquires information which specifiesconditions for calculating an isosurface from information input by theuser. The UI 6 causes the field calculator 7A to execute the process ofcalculating a scalar field, on the basis of the acquired inputinformation. The UI 6 also causes the surface information 7B to executethe process of generating surface information, on the basis of theacquired input information.

FIG. 11 is a diagram showing the process of the UI according to thefirst embodiment. The UI 6 generates a GUI screen W when an applicationis activated. The GUI screen is displayed in a display region 2A on thedisplay device 2 (see FIG. 1). In FIG. 11, the GUI screen W is displayedin a part of the display region 2A, however, it may be displayed infull-screen in the display region 2A.

In the case where execution of the process to open the point cloud dataDG is specified by input information, the UI 6 displays the point clouddata DG on the GUI screen W. The execution of the process to open thepoint cloud data DG may be specified by dragging and dropping a file ofthe point cloud data DG onto the GUI screen W. The execution of theprocess to open the point cloud data DG may also be specified byselecting a file of the point cloud data DG from a window showing thehierarchy of the files managed by the OS 5.

The information processing device 1 (see FIG. 1) includes an imagegenerator 13. The image generator 13 generates a point cloud image Im1,using the point cloud data DG. In the present embodiment, a point cloudimage (for example, the point cloud image Im1) is an image in whichpoints of predetermined sizes are arranged at coordinates on the imagecorresponding to the coordinates of each point data of the point clouddata DG. All of the points displayed in the image may be of the samesize, or some of the points may be of the size different from that ofother points. A point cloud image (for example, the point cloud imageIm1) is, for example, an image representing a spatial distribution of aplurality of points included in the point cloud data DG. The point clouddata DG may be generated preliminarily, or may be generated by theinformation processing device 1. For example, the information processingdevice 1 generates, by means of a rendering process, the point cloudimage Im1 when the distribution of the points is viewed from apredetermined viewpoint.

In a STORM, a plurality of fluorescent images are acquired by activatinga fluorescent substance and irradiating the activated fluorescentsubstance with an excitation light. The data of the plurality offluorescence images are input to the information processing device 1.The image generator 13 calculates position information of thefluorescent substance in each fluorescent image, and generates pointcloud data DG, using a plurality of pieces of the calculated positioninformation. The image generator 13 generates the point cloud image Im1representing the point cloud data DG. In the case of a two-dimensionalSTORM, the image generator 13 calculates two-dimensional positioninformation of a fluorescent substance and generates point cloud data DGincluding a plurality of pieces of two-dimensional data. In the case ofa three-dimensional STORM, the image generator 13 calculatesthree-dimensional position information of a fluorescent substance andgenerates point cloud data DG including a plurality of pieces ofthree-dimensional data. N-dimensional data such as the two-dimensionaldata and the three-dimensional data mentioned above can be acquired bymeans of not only a microscope but also CT scanning or the like. In atechnique for processing N-dimensional data, for example, it isdesirable that the user can with ease comprehend a shape represented byN-dimensional data.

The output controller 12 outputs the data of the point cloud image Im1to the display device 2 via the OS 5. The display device 2 displays thepoint cloud image Im1 on the basis of the data output from the OS 5. Forexample, when the pointer P is detected as having performed a drag whilebeing placed over the point cloud image Im1, the UI 6 can display thepoint cloud image Im1 as viewed from a changed viewpoint.

The UI 6 acquires information specifying a region AR1 (for example, (anROI)) on the point cloud image Im1 as user input information. Forexample, when a left-click is detected as having been performed whilethe pointer P is being placed over the point cloud image Im1, the UI 6sets a vertex of the rectangular region AR1 at the position of thepointer P. After having set the vertex of the rectangular region AR1,when the pointer P is detected as having performed a drag while beingplaced over the point cloud image Im1, the UI 6 sets the region AR1 withthe vertex thereof being at the destination of the pointer P. The outputcontroller 12 outputs the data of an image showing the set region AR1 tothe display device 2 via the OS 5. The display device 2 displays theimage showing the region AR1 being superposed on the point cloud imageIm1.

The input controller 11 accepts user input information which indicatesto conclusively determine the region AR1. When the input informationindicating to conclusively determine the region AR1 is determined ashaving been accepted, the input controller 11 displays in the GUI screenW a point cloud image Im2 which enlarges the region AR1. For example,the information processing device 1 generates the point cloud image Im2by means of a rendering process. The output controller 12 outputs thedata of the point cloud image Im2 to the display device 2 via the OS 5.The display device 2 displays the point cloud image Im2 on the GUIscreen W.

FIG. 12 is a diagram showing a setting process using the UI according tothe first embodiment. In the present embodiment, the field calculator 7Acalculates, under preliminarily set calculation conditions, a functionwhich represents a scalar field for the region AR1 specified in FIG. 11.The surface generator 7B generates, under the preliminarily setcalculation conditions, surface information for the region AR1 specifiedin FIG. 11, on the basis of the calculation result of the fieldcalculator 7A. The output controller 12 displays in the GUI screen W arestored surface image Im3 representing the surface information obtainedunder the preset calculation conditions. In the present embodiment, therestored surface image (the restored surface image Im3) is an imagewhich represents the shape of an object corresponding to a regionsurrounded by an isosurface. The restored surface image (for example,the restored surface image Im3) is, for example, an image whichrepresents a surface restored (estimated) by the surface generator 7B asa surface of an object represented by the point cloud data DG.

The user can set parameters related to generating surface information,using the setting screen (for example, the GUI screen W). Examples ofthe parameters related to generating surface information include thescalar value of an isosurface (“iso value” in FIG. 12), the radius of aregion covered by a local field (“radius (nm)”) in FIG. 12), and theresolution d (“resolution (nm)” in FIG. 12). For example, the user can,using the setting screen (for example, the GUI screen W), set the valueof “iso value” as a first parameter related to generating surfaceinformation, the value of “radius (nm)” as a second parameter related togenerating surface information, and the value of “resolution (nm)” as athird parameter related to generating surface information. The user neednot set at least one of the first to third parameters.

The setting screen (for example, the GUI screen W) includes firstsetting sections (denoted by reference signs Q1, Q2, and Q3 in FIG. 12)to set a scalar value (for example, “iso value”). The UI 6 acquires, asinput information, information specifying a scalar value (for example, aspecified value iso in FIG. 8A to FIG. 8C) in a scalar field. Forexample, the output controller 12 outputs to the GUI screen W an “isovalue” representing a scalar value as an item specified in inputinformation. The output controller 12 displays an input window Q1, aslider Q2, and buttons Q3 at positions preliminarily set with respect tothe position displayed as “iso value”. The output controller 12 displaysa preliminarily set scalar value (for example, an initial value) in theinput window Q1. In FIG. 12, “10” is displayed as the specified value ofthe preliminarily set scalar value.

When the pointer P is detected as having performed a drag while beingplaced over the slider Q2, the input controller 11 calculates thespecified value of the scalar value on the basis of the movement amountof the pointer P. The output controller 12 updates the value displayedin the input window Q1 to the specified value calculated by the inputcontroller 11. The image processor 7 (for example, the field calculator7A) calculates the scalar field on the basis of the point cloud data DG.The image processor 7 (for example, the surface generator 7B) calculatesthe isosurface of the scalar field on the basis of the scalar value setin the first setting sections and generates surface information.

The user can also input the specified value for the scalar value asinput information by operating a keyboard. For example, when aleft-click is detected as having been performed while the point P isbeing placed over the input window Q1, the input controller 11 acquiresa numerical value input using the keyboard. The output controller 12updates the value displayed in the input window Q1 to the value acquiredby the input controller 11 as a numerical value input using thekeyboard.

The buttons Q3 are assigned with a process of increasing or decreasingthe value displayed in the input window Q1. The input controller 11calculates the specified value of the scalar value on the basis of thenumber of times a left-click is detected as having been performed whilethe pointer P is being placed over the button Q2 or the length of timeduring which a left-click is performed. The output controller 12 updatesthe value displayed in the input window Q1 to the specified valuecalculated by the input controller 11.

The setting screen (for example, the GUI screen W) includes settingsections (denoted by reference signs Q4, Q5, and Q6 in FIG. 12) to setinformation related to a local scalar field (for example, “radius(nm)”). “Radius (nm)” corresponds to information related to a thresholdvalue of a function in which the value is constant within a region wherethe distance from each point data is equal to or greater than thethreshold value. For example, f1 in FIG. 2C is a function in which thevalue is constant in a region where the distance from each point data isequal to or greater than the threshold value r1, and in such a case, thevalue of “radius (nm)” is used for the threshold value r1. For example,f5 in FIG. 4B is a function in which the value is constant in the regionwhere the distance from each point data is equal to or greater than thethreshold value r2, and in such a case, the value of “radius (nm)” isused for the threshold value r2. “Radius (nm)” may be a piece ofinformation related to a coefficient of a function representing a localfield. “Radius (nm)” may be used for the coefficient included in afunction in which the value changes continuously (for example, f4 inFIG. 4A, or f5 in FIG. 4B). For example, the shape of a function (forexample, the slope at each point) may be changed by changing the valuespecified by “radius (nm)”. “Radius (nm)” may be a piece of informationrelated to both the threshold value and the coefficient of the abovefunction. For example, for f5 in FIG. 4B, if “radius (nm)” is reduced,r2 may be reduced and the slope of f5 may be increased.

The UI 6 acquires, as input information, information which specifies theradius of a local field (for example, the radius r in FIG. 10A and FIG.10B). The UI 6 acquires, as information specifying the radius of a localfield, information which specifies the radius centered on the sourceorigin thereof. For example, the output controller 12 outputs to the GUIscreen W “radius (nm)” representing the radius of a region covered by alocal field as an item specified in the input information. The outputcontroller 12 displays an input window Q4, a slider Q5, and buttons Q6at positions preliminarily set with respect to the position displayed as“radius (nm)”. The output controller 12 displays a preliminarily setradius (for example, an initial value) in the input window Q4. In FIG.12, “100” is displayed as the preliminarily set radius.

When the pointer P is detected as having performed a drag while beingplaced over the slider Q5, the input controller 11 calculates thespecified value of the radius on the basis of the movement amount of thepointer P. The output controller 12 updates the value displayed in theinput window Q4 to the specified value calculated by the inputcontroller 11. The image processor 7 (for example, the field calculator7A) calculates the scalar field on the basis of information related tothe local scalar field (for example, “radius (nm)”) set in the settingsections denoted by reference signs Q4, Q5, and Q6 in FIG. 12.

The user can also input the specified value for the scalar value asinput information by operating a keyboard. For example, when aleft-click is detected as having been performed while the point P isbeing placed over the input window Q4, the input controller 11 acquiresa numerical value input using the keyboard. The output controller 12updates the value displayed in the input window Q4 to the value acquiredby the input controller 11 as a numerical value input using thekeyboard.

The buttons Q6 are assigned with a process of increasing or decreasingthe value displayed in the input window Q4. The input controller 11calculates the specified value of the radius on the basis of the numberof times a left-click is detected as having been performed while thepointer P is being placed over the button Q6 or the length of timeduring which a left-click is performed. The output controller 12 updatesthe value displayed in the input window Q4 to the specified valuecalculated by the input controller 11.

The setting screen (for example, the GUI screen W) includes settingsections (denoted by reference signs Q7, Q8, and Q9 in FIG. 12) to setinformation related to the interval between grid points (for example,“resolution”). The UI 6 acquires, as input information, informationwhich specifies an interval between grid points (for example, theresolution d in FIG. 10A and FIG. 10B) in the process of generatingsurface information. For example, the output controller 12 outputs tothe GUI screen W “resolution (nm)” representing the resolution d as anitem specified in the input information. The output controller 12displays an input window Q7, a slider Q8, and buttons Q9 at positionspreliminarily set with respect to the position displayed as “resolution(nm)”. The output controller 12 displays a preliminarily set resolution(for example, an initial value) in the input window Q7. In FIG. 12, “25”is displayed as the preliminarily set resolution.

When the pointer P is detected as having performed a drag while beingplaced over the slider Q8, the input controller 11 calculates thespecified value of the resolution d on the basis of the movement amountof the pointer P. The output controller 12 updates the value displayedin the input window Q7 to the specified value calculated by the inputcontroller 11. The input information acquirer (the input controller 11)acquires information (for example, the value of the specified“resolution”) related to the interval between the grid points input bythe inputter (the input device 3). The surface generator 7B discretizesscalar fields at grid points on the basis of the input informationrelated to the interval between the grid points.

The user can also input the specified value for the resolution d asinput information by operating a keyboard. For example, when aleft-click is detected as having been performed while the point P isbeing placed over the input window Q7, the input controller 11 acquiresa numerical value input using the keyboard. The output controller 12updates the value displayed in the input window Q7 to the value acquiredby the input controller 11 as a numerical value input using thekeyboard.

The buttons Q9 are assigned with a process of increasing or decreasingthe value displayed in the input window Q7. The input controller 11calculates the specified value of the resolution d on the basis of thenumber of times a left-click is detected as having been performed whilethe pointer P is being placed over the button Q9 or the length of timeduring which a left-click is performed. The output controller 12 updatesthe value displayed in the input window Q9 to the specified valuecalculated by the input controller 11.

“Thumbnail”, “Advanced”, “OK”, “Cancel”, and “Apply” buttons aredisplayed on the GUI screen W. A process of displaying other settingitems is assigned to the “Advanced” button.

FIG. 13 is a diagram showing a setting process using the UI according tothe first embodiment. The setting screen (for example, the GUI screen W)includes second setting sections (denoted by reference signs Q11 and Q12in FIG. 13) to set information related to the function representing alocal scalar field. When a left-click is detected as having beenperformed while the pointer P is being placed over the “Advanced”button, the input controller 11 causes the output controller 12 toexecute the process of displaying other setting items. For example, asitems specified in input information, the output controller 12 outputsto the GUI screen W “Function” which indicates a function representing alocal field as an item specified in the input information.

The information related to a function mentioned above is informationrelated to types of function (for example, “Rectangle”, “Gaussian”,“Edit” of “Function” in FIG. 13). The output controller 12 displays“Rectangle”, a button Q11, “Gaussian”, a button Q12, and “Edit” atpositions preliminarily set with respect to the position displayed as“Function”. The output controller 12 causes the display device 2 todisplay “Rectangle” and “Gaussian” as images showing candidates of thefunction representing a local field.

The button Q11 is assigned with a process of specifying a square-wavefunction (see FIG. 2C) as a function representing a local field. Thebutton Q12 is assigned with a process of specifying a functionrepresenting a Gaussian distribution (see FIG. 4A to FIG. 4B) as afunction representing a local field. “Edit” is assigned with a processof editing a function representing a local field. For example, “Edit” isassigned with a process of reading a file which defines a functionrepresenting a local field.

The input information acquirer (the input controller 11) acquiresinformation related to a function representing the local scalar fieldinput by the inputter (the input device 3). The field calculator 7Acalculates the scalar field, using the information related to thefunction acquired by the input information acquirer (the inputcontroller 11). As information which represents the specified functionamong the function candidates, the input controller 11 acquires inputinformation input using the display device 2 (the GUI screen W).

The image processor 7 (for example, the field calculator 7A) generates alocal scalar field on the basis of one or more pieces of point data ofthe point cloud data DG on the basis of the function set in the secondsetting sections. The image processor 7 (for example, the fieldcalculator 7A) calculates the scalar field on the basis of local scalarfields by superposing the local scalar fields. The image processor 7(for example, the surface generator 7B) calculates the isosurface of thescalar field on the basis of the local scalar fields and generatessurface information.

The information related to the function mentioned above is informationrelated to types of function. The information related to types offunction mentioned above includes at least one of a function in whichthe value changes stepwise with respect to the distance from each pointdata, a function in which the value continuously changes with respect tothe distance from each point data, and a function in which the valuedecreases with respect to the distance from each point data. Theinformation related to types of function mentioned above may be a pieceof information related to a threshold value of a function in which thevalue is constant in a region where the distance from each point data isequal to or greater than the threshold value.

The setting screen (for example, the GUI screen W) includes thirdsetting sections (denoted by reference signs Q13 and Q14 in FIG. 13) toset information related to filters. As an item specified in the inputinformation, the output controller 12 outputs to the GUI screen W“Filter” which indicates a filter used in calculating a functionrepresenting a scalar field. The output controller 12 displays“Gaussian”, a button Q13, “Laplacian”, a button Q14, and “Edit” atpositions preliminarily set with respect to the position displayed as“Filter”. The output controller 12 causes the display device 2 todisplay “Gaussian” and “Laplacian” as images showing the filtercandidates.

The button Q13 is assigned with a process of specifying a Gaussianfilter as a filter. The button Q14 is assigned with a process ofspecifying a Laplacian filter as a filter. “Edit” is assigned with aprocess of editing the function representing a filter. For example,“Edit” is assigned with a process of reading a file which defines thefunction representing a filter. The field calculator 7A superposes localscalar fields and applies a filter thereto to calculate the functionrepresenting a scalar field. The image processor 7 (for example, thefield calculator 7A) superposes local scalar fields and applies thefilter thereto on the basis of the information related to the filter setin the third setting sections. The image processor 7 (for example, thefield calculator 7A) generates a scalar field by applying a filter tothe function in which local scalar fields are superposed.

The input information acquirer (the input controller 11) acquiresinformation related to the filter input by the inputter (the inputdevice 3). The information related to the filter is a piece ofinformation which specifies the function representing the filter (forexample, “Gaussian”, “Laplacian”, and “Edit” of “Filter” in FIG. 13).The field calculator 7A calculates a scalar field, using the informationrelated to the filter acquired by the input information acquirer (theinput controller 11).

“Apply” is assigned with a process of reflecting the information of anitem specified in the input information. “Cancel” is assigned with aprocess of reverting the information of the changed item. “OK” isassigned with a process of completing the setting. When the settingprocess described above is completed, the field calculator 7A calculatesa function which represents a scalar field, using the function(“Function”), the radius (“radius (nm)”), and the filter (“Filter”)specified in the input information. The surface generator 7B calculatessurface information, on the basis of the scalar value (“iso value”) andthe resolution (resolution (nm)) specified in the input information, andthe function representing the scalar field calculated by the fieldcalculator 7A.

In the setting process, at least one of the scalar value (“iso value”),the radius (“radius (nm)”), and the resolution (resolution (nm)) can beset, and preliminarily set values (fixed values) may be used for theother parameters. In the setting process, one or both of the radius(“radius (nm)”) and the resolution (resolution (nm)) need not be set.For example, the surface generator 7B may generate surface information,using a preliminarily set value (for example, a fixed value) for one orboth of the radius (“radius (nm)”) and the resolution (resolution (nm)).In the setting process, parameters other than the scalar value (“isovalue”), the radius (“radius (nm)”), and the resolution (resolution(nm)) may be set.

In an additional setting process (“Advanced”), one or both of“Rectangle” and “Gaussian” need not be presented as candidates for afunction (“Function”) representing a local field. In the additionalsetting process (“Advanced”), a candidate different from “Rectangle” and“Gaussian” may be presented as a candidate for a function (“Function”)representing a local field. In the additional setting process(“Advanced”), the process of specifying the function representing alocal field need not be executed.

In the additional setting process (“Advanced”), one or both of“Gaussian” and “Laplacian” need not be presented as candidates for thefunction (“filter”) representing a filter. In the additional settingprocess (“Advanced”), a candidate different from “Gaussian” and“Laplacian” may be presented as a candidate for the function (“filter”)representing a filter. In the additional setting process (“Advanced”),the process of specifying the function representing a filter need not beexecuted. Furthermore, the additional setting process (“Advanced”) neednot be executed.

In the present embodiment, the surface generator 7B generates surfaceinformation, using, for example, a plurality of calculation conditions.The output controller 12 causes the display device 2 to display imagesrepresenting surface information generated by the surface generator 7B.For example, when a left-click is detected as having been performedwhile the pointer P is being placed over “Thumbnail”, the inputcontroller 11 causes the output controller 12 to output the images(shown later in FIG. 14 to FIG. 16) representing the surface information(see FIG. 12 and FIG. 13).

In the case where at least one of the calculation conditions (forexample, the specified value iso of the scalar value, the resolution d,and the radius r) is changed, the displayed images may be changed inreal time. That is to say, even without “Apply” or “OK” being clicked(pressed), the surface generator 7B may generate surface information inreal time, using the changed calculation conditions, and the outputcontroller 12 may cause the display device 2 to display the imagesrepresenting surface information and generated by the surface generator7B.

FIG. 14 is a diagram showing images displayed by the UI according to thefirst embodiment. In FIG. 14, a plurality of restored surface imagesIm4, for which the calculation conditions differ, are displayed in aform of a table on the GUI screen W. The horizontal axis of this tablecorresponds to the specified value of scalar value (“iso value”). Forexample, among the plurality of restored surface images Im4, a restoredsurface image Im4 a and a restored surface image Im4 b in the samecolumn are of the same specified value of scalar value (for example,“10”). Among the plurality of images Im4, the image Im4 a and an imageIm4 c in different columns are of different specified values of scalarvalue (for example, “10”, “25”).

The output controller 12 causes the display device 2 (the GUI screen W)to display, in an arranged manner, a first restored surface image (forexample, the image Im4 a) representing surface information calculatedusing a first calculation condition, and a second restored surface image(for example, an image Im4 c) representing surface informationcalculated using a second calculation condition, which is different fromthe first calculation condition. The first calculation condition differsfrom the second calculation condition in the scalar value in scalarfield.

The vertical axis of the table corresponds to the specified value ofresolution (“resolution”). For example, the image Im4 a and the imageIm4 c in the same row are of the same specified value of resolution (forexample, “25”). The restored surface image Im4 a and the restoredsurface image Im4 b in different rows are of different specified valuesof resolution (for example, “25”, “50”). The output controller 12 causesthe display device 2 (the GUI screen W) to display, in an arrangedmanner, the first restored surface image (for example, the image Im4 a)representing surface information calculated using the first calculationcondition, and the second restored surface image (for example, an imageIm4 b) representing surface information calculated using the secondcalculation condition, which is different from the first calculationcondition. The first calculation condition differs from the secondcalculation condition in the specified value of resolution.

FIG. 15 is a diagram showing images displayed by the UI according to thefirst embodiment. In FIG. 15, a plurality of restored surface imagesIm4, for which calculation conditions differ, are displayed in a form ofa table on the GUI screen W. The horizontal axis of this tablecorresponds to the specified value of scalar value (“iso value”). Forexample, among the plurality of restored surface images Im4, a restoredsurface image Im4 d and a restored surface image Im4 e in the samecolumn are of the same specified value of scalar value (for example,“10”). Among the plurality of images Im4, the restored surface image Im4d and a restored surface image Im4 f in different columns are ofdifferent specified values of scalar value (for example, “10”, “25”).The output controller 12 causes the display device 2 (the GUI screen W)to display, in an arranged manner, the first restored surface image (forexample, the restored surface image Im4 d) representing surfaceinformation calculated using the first calculation condition, and thesecond restored surface image (for example, the restored surface imageIm4 f) representing surface information calculated using the secondcalculation condition, which is different from the first calculationcondition. The first calculation condition differs from the secondcalculation condition in the scalar value in scalar field.

The vertical axis of the table corresponds to the specified value ofradius (“radius”). For example, the restored surface image Im4 d and therestored surface image Im4 f in the same row are of the same specifiedvalue of radius (for example, “75”). The restored surface image Im4 dand the restored surface image Im4 e in different rows are of differentspecified values of radius (for example, “75”, “100”). The outputcontroller 12 causes the display device 2 (the GUI screen W) to display,in an arranged manner, the first restored surface image (for example,the restored surface image Im4 d) representing restored surfaceinformation calculated using the first calculation condition, and thesecond restored surface image (for example, the restored surface imageIm4 e) representing surface information calculated using the secondcalculation condition, which is different from the first calculationcondition. The first calculation condition differs from the secondcalculation condition in the specified value of radius.

FIG. 16 is a diagram showing images displayed by the UI according to thefirst embodiment. In FIG. 16, a plurality of restored surface imagesIm4, for which calculation conditions differ, are displayed in a form ofa table on the GUI screen W. The horizontal axis of the tablecorresponds to the specified value of resolution (“resolution”). Forexample, among the plurality of restored surface images Im4, a restoredsurface image Im4 g and a restored surface image Im4 h in the samecolumn are of the same specified value of resolution (for example,“25”). Among the plurality of images Im4, the image Im4 g and an imageIm4 i in different columns are of different specified values ofresolution (for example, “25”, “50”). The output controller 12 causesthe display device 2 (the GUI screen W) to display, in an arrangedmanner, the first restored surface image (for example, the image Im4 g)representing surface information calculated using the first calculationcondition, and the second restored surface image (for example, the imageIm4 i) representing surface information calculated using the secondcalculation condition, which is different from the first calculationcondition. The first calculation condition differs from the secondcalculation condition in the specified value of resolution.

The vertical axis of the table corresponds to the specified value ofradius (“radius”). For example, the restored surface image Im4 g and therestored surface image Im4 i in the same row are of the same specifiedvalue of radius (for example, “75”). The restored surface image Im4 gand the restored surface image Im4 h in different rows are of differentspecified values of radius (for example, “75”, “100”). The outputcontroller 12 causes the display device 2 (the GUI screen W) to display,in an arranged manner, the first restored surface image (for example,the restored surface image Im4 g) representing surface informationcalculated using the first calculation condition, and the secondrestored surface image (for example, the restored surface image Im4 h)representing restored surface information calculated using the secondcalculation condition, which is different from the first calculationcondition. The first calculation condition differs from the secondcalculation condition in the specified value of radius.

The information processing device 1 displays the plurality of restoredsurface images Im4 obtained under different calculation conditions inthe arranged manner as described above. The user can, for example,compare the plurality of restored surface images Im4 and selectcalculation conditions suitable for extracting a target structure. Forexample, the surface generator 7B generates surface informationcandidates under a plurality of conditions with different parametervalues related to generating surface information (for example,“resolution (nm)”, “iso value”), and generates a plurality of imagesrepresenting the surface information candidates (for example, therestored surface images such as the image Im4 a and the image Im4 b inFIG. 14). In the case where an instruction to set parameters is input onthe setting screen (for example, the GUI screen W of FIG. 13) (forexample, where “Thumbnail” is selected), the display controller (theoutput controller 12) causes the display (the display device 2) todisplay the image Im4 a and the image Im4 b as shown in FIG. 14. Thedisplay controller (the output controller 12) displays the image Im4 aand the image Im4 b as samples of images representing surfaceinformation.

By selecting an image from the plurality of images (surface informationcandidates) displayed on the display (the display device 2), the usercan specify the parameter value corresponding to the selected image as aparameter related to generating surface information. The image processor7 (the field calculator 7A and the surface generator 7B) generatessurface information, using the parameter value corresponding to theimage selected from the plurality of images displayed on the display(the display device 2). The display controller (the output controller12) causes the display (the GUI screen W of FIG. 13) to display theparameter value corresponding to the selected image, and an image whichrepresents the surface information generated using the parameter valuecorresponding to the selected image. Thus, the display controller (theoutput controller 12) may cause the display (the display device 2) todisplay, in a selectable manner, an image representing first surfaceinformation, an image representing second surface information, an imagerepresenting third surface information, and an image representing fourthsurface information. The display controller (the output controller 12)may cause the setting screen to display the calculation conditioncorresponding to the image selected from the images displayed on thedisplay (the display device 2).

Prior to performing parameter setting using the image candidates of FIG.14, the display controller (the output controller 12) causes the display(for example, the GUI screen W in FIG. 13) to display the first image(for example, the restored surface image Im3 of FIG. 13) generated usingthe first calculation condition of the first parameter. After the firstparameter has been set to the second calculation condition using theimage candidate of FIG. 14, the display controller (the outputcontroller 12) causes the display (for example, the GUI screen W of FIG.13) to display, instead of the first image, the second image (forexample, the image Im4 a of FIG. 14) generated by the image processor 7using the second calculation condition of the second parameter.

In the case where an instruction to set parameters is input on thesetting screen (for example, the GUI screen W of FIG. 13) (for example,where “Thumbnail” is selected), the input controller 11 may accept fromthe user an input specifying the types of parameters to be set. Forexample, in the case where “iso value” and “resolution” are specified asthe types of parameters to be set, as shown in FIG. 14, the outputcontroller 12 may display a plurality of images in which the parametersare different. In the case where “iso value” and “radius” are specifiedas the types of parameters to be set, as shown in FIG. 15, the outputcontroller 12 may display a plurality of images in which the parametersare different. In the case where “radius” and “resolution” are specifiedas the types of parameters to be set, as shown in FIG. 16, the outputcontroller 12 may display a plurality of images in which the parametersare different. There are two types of parameters to be set in FIG. 14 toFIG. 16, however, there may be one type of parameter, or three or moretypes of parameters.

In the case where the first calculation condition is set on the settingscreen (the GUI screen W), the image processor 7 (for example, thesurface generator 7B) may generate the first surface information on thebasis of point cloud data, using the first calculation condition. Thedisplay controller (the output controller 12) may cause the display (thedisplay device 2) to display the image representing the first surfaceinformation. In the case where the second calculation condition is seton the setting screen (the GUI screen W), the image processor 7 (forexample, the surface generator 7B) may generate the second surfaceinformation on the basis of point cloud data, using the secondcalculation condition. The display controller (the output controller 12)may cause the display to display an image representing the first surfaceinformation (for example, the image Im3) and an image representing thesecond surface information (not shown in the drawings). The displaycontroller (the output controller 12) may display the first imagerepresenting the first surface information and the second imagerepresenting the second surface information simultaneously. The displaycontroller (the output controller 12) may display the image representingthe first surface information and the image representing the secondsurface information in a switchable manner.

The image processor 7 (for example, the surface generator 7B) maygenerate a third image using a third calculation condition of the secondparameter, which is different from the first parameter, and may generatea fourth image using a fourth calculation condition of the secondparameter. The display controller may also display the third image andthe fourth image. For example, in the case where the first parameter is“iso value” and the second parameter is “resolution”, the displaycontroller (the output controller 12) may display the third image andthe fourth image as shown in FIG. 14. In the case where the firstparameter is “iso value” and the second parameter is “radius”, thedisplay controller (the output controller 12) may display the thirdimage and the fourth image as shown in FIG. 15. In the case where thefirst parameter is “resolution” and the second parameter is “radius”,the display controller (the output controller 12) may display the thirdimage and the fourth image as shown in FIG. 16.

The above scalar value (“iso value”) may be set on the basis of thedensity (numerical density, crude density) of points in the point setrepresented by the point cloud data DG. For example, there are somecases where a region having a relatively high numerical density ofpoints in the point cloud data DG and a region having a relatively lownumerical density of points are both present. In such a case, it ispossible, by lowering “iso value”, to detect the region having arelatively high numerical density of points and the region having arelatively low numerical density of points. It is also possible, byincreasing “iso value”, to remove the region having a relatively lownumerical density of points to detect (extract) the region having arelatively high numerical density of points. In the case where theregion having a relatively low numerical density of points is a noiseregion, noise can be removed by increasing “iso value”.

In the case where there is an obvious noise (a region with hardly anysurrounding points) which can be recognized by visually observing thepoint cloud image Im1 of FIG. 11, the noise can be preliminarily removedby specifying the noise region on the point cloud image Im1. The imageprocessor 7 may generate surface information after the noise has beenremoved. The value of the field near the noise may be reduced byapplying a filter (for example, a smoothing filter) to the field, tothereby remove the noise.

The display device 2 may be able to display an image three-dimensionallyby means of a parallax image. In such a case, the output controller 12may cause the display device 2 to display a parallax image. A printerwhich forms a three-dimensional structure may be connected to theinformation processing device 1. In such a case, the output controller12 may output three-dimensional shape information obtained from surfaceinformation to the printer to form the three-dimensional structure.

Next, an information processing method according to the embodiment willbe described, on the basis of the operation of the informationprocessing device 1 described above. FIG. 17 is a flowchart showing theinformation processing method according to the first embodiment. In StepS1, the information processing device 1 sets parameters (calculationconditions). For example, as described in FIG. 11 to FIG. 13, the UI 6sets a parameter used for calculating a function representing a scalarfield FS (for example, “radius”), the function representing local fields(for example, “function”), and the function representing a filter (forexample, “filter”). The UI 6 sets parameters (for example, “resolution”,“iso value”) used for generating surface information. For example, inStep S2, the input controller 11 acquires information which specifiesthe scalar value. In Step S3, the input controller 11 acquiresinformation which specifies the resolution. In Step S4, the inputcontroller 11 acquires information which specifies the radius.

The processes of Step S2 to Step S4 may be executed in any order. When apreliminarily set value is used as the resolution, the process of StepS3 need not be executed. When a preliminarily set value is used as theradius, the process of Step S4 need not be executed.

In Step S5, the field calculator 7A calculates a function whichrepresents scalar fields (see FIG. 3A to FIG. 3D). The field calculator7A calculates a function representing scalar fields, using theinformation acquired by the input controller 11 in Step S1. The fieldcalculator 7A stores the calculated function representing scalar fieldsin the memory storage 9. In Step S6, the surface generator 7B generatessurface information representing an isosurface (see FIG. 5, FIG. 6A toFIG. 6E, FIG. 7, FIG. 8A to FIG. 8C, FIG. 9A to FIG. 9D, FIG. 10A, andFIG. 10B). The surface generator 7B generates the surface informationusing the information acquired by the input controller 11 in Step S1.The surface generator 7B stores the generated surface information in thememory storage 9.

In Step S7, the information processing device 1 outputs an imagerepresenting the surface information (see FIG. 14 to FIG. 17). Forexample, the output controller 12 outputs image data to the displaydevice 2, on the basis of input information from the user who specifiesto output the image representing the surface information. The outputcontroller 12 reads out the surface information stored in the memorystorage 9 by the surface generator 7B, and causes the display device 2(for example, the GUI screen W) to display the image representing thesurface information. The output controller 12 displays the image showingthe shape of a region surrounded by an isosurface as an image showingthe surface information.

The information processing method according to the embodiment includes:causing the display to display the setting screen to set the calculationconditions used for generating surface information; generating surfaceinformation on the basis of point cloud data, using the calculationconditions set in the setting screen; and causing the display to displayan image representing the generated surface information. The informationprocessing method according to the embodiment may include: generatingthe first surface information, using the first calculation condition ofthe first parameter on the basis of the point cloud data; generating thesecond surface information, using the second calculation condition ofthe first parameter on the basis of the point cloud data; and causingthe display to display an image representing the first surfaceinformation and an image representing the second surface information.The information processing method according to the embodiment mayinclude: calculating a scalar field on the basis of point cloud data;acquiring a scalar value input by the inputter; and calculating theisosurface of a scalar field on the basis of the acquired scalar valueto generate surface information.

Second Embodiment

Next, a second embodiment will be described. In the present embodiment,the same reference signs are given to the same configurations as thosein the embodiment described above, and the descriptions thereof will beomitted or simplified. FIG. 18 is a diagram showing an informationprocessing device according to a second embodiment. A processor 15includes a computation unit 18. The computation unit 18 computes atleast one of: the area of an isosurface; the volume of a spacesurrounded by an isosurface; the linear distance between two points onan isosurface; the geodetic distance between two points on anisosurface; the distance between the centroid of a structure in aspecified first region and the centroid of a structure in a specifiedsecond region; and the degree of similarity between a first isosurfaceand a second isosurface represented by surface information.

FIG. 19 to FIG. 24 are diagrams showing processes of the processor andthe UI according to the second embodiment. First, the process ofcomputing distance by means of the computation unit 18 will bedescribed. In FIG. 19, the GUI screen W is displaying a restored surfaceimage Im5 representing the surface information generated by the surfacegenerator 7B, and a button Q21. The button Q21 is assigned with aprocess of acquiring, as input information, a condition for theprocessing performed by the computation unit 18. The output controller12 displays the button Q21 in the GUI screen W on the basis of GUIinformation. When a left-click is detected as having been performedwhile the pointer P is being placed over the button Q21, the inputcontroller 11 causes the output controller 12 to generate a window W1.

A button Q22, a button Q23, and an image Im6 are displayed in the windowW1. The button Q22 is assigned with a process of causing the computationunit 18 to execute distance calculation. When a left-click is detectedas having been performed while the pointer P is being placed over thebutton Q22, the input controller 11 accepts, as input information,information which specifies a position in the image Im5. When aleft-click is detected as having been performed while the pointer P isbeing placed over the image Im5, the input controller 11 acquires theposition of the pointer P as the position of a first point specified bythe user. The input controller 11 causes the output controller 12 todisplay a mark Q24 at the acquired position of the pointer P. Similarly,when a left-click is detected as having been performed while the pointerP is being placed over the image Im5, the input controller 11 acquiresthe position of the pointer P as the position of a second pointspecified by the user. The input controller 11 causes the outputcontroller 12 to display a mark Q25 at the acquired position of thepointer P.

When the position of the first point (see the mark Q24) and the positionof the second point (see the mark Q25) have been acquired, the inputcontroller 11 causes the computation unit 18 to compute the lineardistance between the first point and the second point. The computationunit 18 reads out surface information stored in the memory storage 9 tocompute the linear distance between the first point and the secondpoint. An image Im6 is an image showing computation results of thecomputation unit 18. Here, it is assumed that the linear distancebetween the first point and the second point computed by the computationunit 18 is 80 nm. The output controller 12 reflects a computation resultof the computation unit 18 (for example, “80 (Straight line)”) on theimage Im6.

As shown in FIG. 20, the output controller 12 displays on the image Im5a line Q26 connecting the mark Q24 and the mark Q25 and a computationresult (for example, “80 nm”) of the computation unit 18. When aright-click is detected as having been performed while the pointer P isbeing placed over the line Q26, the input controller 11 causes theoutput controller 12 to execute a process of generating a window W2. Abutton labeled “Straight line distance” and a button labeled “Geodesicdistance” are displayed in the window W2. The “Straight line distance”button is assigned with a process of specifying to compute a lineardistance. The “Geodesic distance” button is assigned with a process ofspecifying to compute a geodesic distance.

When a left-click is detected as having been performed while the pointerP is being placed over “Geodesic distance”, the input controller 11causes the computation unit 18 to compute the geodetic distance betweenthe first point and the second point. The computation unit 18 reads outsurface information stored in the memory storage 9 to compute thegeodetic distance between the first point and the second point. Here, itis assumed that the geodetic distance between the first point and thesecond point computed by the computation unit 18 is 100 nm. The outputcontroller 12 displays on the image Im5 a geodesic line Q27 connectingthe mark Q24 and the mark Q25 and a computation result (for example,“100 nm”) of the computation unit 18. The output controller 12 reflectsthe computation result of the computation unit 18 on the image Im6 shownin the lower figure of FIG. 19. For example, the output controller 12displays “100 nm (Geodesic line)” in the “Length” column.

Next, a process of computing a volume and an area by means of thecomputation unit 18 will be described. In FIG. 21, the GUI screen W isdisplaying an image Im7 representing the surface information generatedby the surface generator 7B, and a button Q21. The button Q21 isassigned with a process of acquiring, as input information, a conditionfor the processing performed by the computation unit 18. The outputcontroller 12 displays the button Q21 in the GUI screen W on the basisof GUI information. When a left-click is detected as having beenperformed while the pointer P is being placed over the button Q21, theinput controller 11 causes the output controller 12 to generate thewindow W1.

The button Q22, the button Q23, and the image Im6 are displayed in thewindow W1. The button Q23 is assigned with a process of causing thecomputation unit 18 to execute computation of a volume and an area. Whena left-click is detected as having been performed while the pointer P isbeing placed over the button Q23, the input controller 11 accepts, asinput information, information which specifies a region AR2 (see FIG.22) in the image Im7.

When a left-click is detected as having been performed while the pointerP is being placed over the image Im7, the input controller 11 acquiresthe position of the pointer P as the position of a vertex of the regionspecified by the user. When the pointer P is detected as havingperformed a drag while being placed over the image Im7, the inputcontroller 11 determines the region AR2 on the basis of the movementamount of the pointer P. The input controller 11 causes the computationunit 18 to compute the volume of a space (a region) surrounded by anisosurface in the determined region AR2. The computation unit 18 readsout surface information stored in the memory storage 9 and computes thevolume of the space surrounded by the isosurface in the region AR2. Theinput controller 11 causes the computation unit 18 to compute the areaof the isosurface in the region AR2. The area of an isosurfacecorresponds to the surface area of an object (a region) surrounded bythe isosurface. The computation unit 18 reads out surface informationstored in the memory storage 9 and computes the volume of the area ofthe isosurface in the region AR2.

The image Im6 is an image showing computation results of the computationunit 18. Here, it is assumed that the volume computed by the computationunit 18 is 4186 nm³. The output controller 12 reflects the computationresult of the computation unit 18 on the image Im6, and displays “4186”in the “Volume” column. The output controller 12 reflects thecomputation result of the computation unit 18 on the image Im7, anddisplays “4186 nm³” near the region AR2. It is assumed that the areacomputed by the computation unit 18 is 1256 nm². The output controller12 reflects the computation result of the computation unit 18 on theimage Im6, and displays “1256” in the “Surface area” column.

As shown in the lower figure of FIG. 22, when a right-click is detectedas having been performed while the pointer P is being placed over theouter periphery of the region AR2, the input controller 11 causes theoutput controller 12 to generate a window W3. “Volume”, “Surface area”,and “Plot centroid position” are displayed in the window W3. “Volume” isassigned with a process of displaying the volume computed by thecomputation unit 18 on the image Im6. “Surface area” is assigned with aprocess of displaying the area computed by the computation unit 18 onthe image Im7. When a left-click is detected as having been performedwhile the pointer P is being placed over “Surface area”, the inputcontroller 11 displays “1256 nm²” near the region AR2.

As shown in FIG. 23, when a left-click is detected as having beenperformed while the pointer P is being placed over “Plot centroidposition”, the input controller 11 causes the computation unit 18 tocompute the centroid of a space surrounded by an isosurface in theregion AR2. The computation unit 18 reads out surface information storedin the memory storage 9 and calculates the position of the centroidcorresponding to the region AR2. The output controller 12 displays amark Q31 at a position on the image Im7 corresponding to the centroidcomputed by the computation unit 18.

The output controller 12 accepts information which specifies a regionAR3 (see the lower figure of FIG. 23). The user can specify the regionAR3 in a manner similar to that in the case of specifying the regionAR2. When the region AR3 is specified, the input controller 11 causesthe computation unit 18 to compute a centroid corresponding to theregion AR3 as with the case of the region AR2. The output controller 12displays a mark Q32 at a position on the image Im7 corresponding to thecentroid computed by the computation unit 18.

The input controller 11 causes the output controller 12 to generate thewindow W1 shown in FIG. 24. When a left-click is detected as having beenperformed while the pointer P is being placed over the button Q22, theinput controller 11 causes the computation unit 18 to compute the lineardistance between the centroid (the mark Q31) corresponding to the regionAR2 and the centroid (the mark Q32) corresponding to the region AR3. Thecomputation unit 18 reads out surface information stored in the memorystorage 9 to compute the linear distance between the centroids. Here, itis assumed that the linear distance between the centroids computed bythe computation unit 18 is 160 nm. The output controller 12 displays onthe image Im7 a line Q33 connecting the mark Q31 and the mark Q32 andthe computation result (for example, “160 nm”) of the computation unit18. The output controller 12 reflects the computation result of thecomputation unit 18 on the image Im6 in the window W1. For example, theoutput controller 12 displays “160 nm (Centroid)” in the “Length”column.

In the embodiment described above, the information processing device 1includes, for example, a computer system. The information processingdevice 1 reads out an information processing program stored in thememory storage 9, and executes various processes in accordance with theinformation processing program. This information processing programcauses a computer to execute processes of: causing the display todisplay the setting screen to set the calculation conditions used forgenerating surface information; generating surface information on thebasis of point cloud data, using the calculation conditions set in thesetting screen; and causing the display to display an image representingthe generated surface information. The information processing programaccording to the embodiment may be a program which causes a computer toexecute processes of: generating the first surface information, usingthe first calculation condition of the first parameter on the basis ofthe point cloud data; generating the second surface information, usingthe second calculation condition of the first parameter on the basis ofthe point cloud data; and causing the display to display an imagerepresenting the first surface information and an image representing thesecond surface information. The information processing program accordingto the embodiment may be a program which causes a computer to executeprocesses of: calculating a scalar field on the basis of point clouddata; acquiring a scalar value input by the inputter; and calculatingthe isosurface of the scalar field on the basis of the acquired scalarvalue to generate surface information. The information processingprogram according to the embodiment may be recorded and provided on acomputer-readable memory storage medium (for example, a non-transitoryrecording medium, or a non-transitory tangible medium).

Microscope

Next, a microscope according to the embodiment will be described. In thepresent embodiment, the same reference signs are given to configurationssimilar to those in the embodiment described above, and the descriptionsthereof will be omitted or simplified. FIG. 25 is a diagram showing themicroscope according to the embodiment. A microscope 50 includes amicroscope main body 51, the information processing device 1 describedin the embodiment described above, and a control device 52. The controldevice 52 includes a controller 53 which controls each part of themicroscope main body 51, and an image processor 54. At least apart ofthe control device 52 may be provided in (may be built-in) themicroscope main body 51. The controller 53 controls the informationprocessing device 1. At least a part of the control device 52 may beprovided in (may be built-in) the information processing device 1.

The microscope main body 51 detects an image of fluorescence emittedfrom a sample containing a fluorescent substance. The microscope mainbody 51 is, for example, a STORM, a PALM, or the like which uses amethod of single-molecule localization microscopy. In a STORM, aplurality of fluorescent images are acquired by activating a fluorescentsubstance and irradiating the activated fluorescent substance with anexcitation light. The data of the plurality of fluorescence images areinput to the information processing device 1. The image generator 13calculates position information of the fluorescent substance in eachfluorescent image, and generates point cloud data DG, using a pluralityof pieces of the calculated position information. The image generator 13generates the point cloud image Im1 representing the point cloud dataDG. In the case of a two-dimensional STORM, the image generator 13calculates two-dimensional position information of a fluorescentsubstance and generates point cloud data DG including a plurality ofpieces of two-dimensional data. In the case of a three-dimensionalSTORM, the image generator 13 calculates three-dimensional positioninformation of a fluorescent substance and generates point cloud data DGincluding a plurality of pieces of three-dimensional data.

FIG. 26 is a diagram showing the microscope main body according to theembodiment. The microscope main body 51 can be used for bothfluorescence observation of a sample labeled with one type offluorescent substance and fluorescence observation of a sample labeledwith two or more types of fluorescent substances. In the presentembodiment, it is assumed that one type of fluorescent substance (forexample, a reporter dye) is used for labeling. The microscope 50 cangenerate a three-dimensional super-resolution image. For example, themicroscope 50 has a mode for generating a two-dimensionalsuper-resolution image and a mode for generating a three-dimensionalsuper-resolution image, and can switch between the two modes.

The sample may contain live cells or cells that are fixed using a tissuefixative solution such as formaldehyde solution, or may contain tissuesor the like. The fluorescent substance may be a fluorescent dye such ascyanine dye, or a fluorescent protein. The fluorescent dye includes areporter dye that emits fluorescence upon receiving excitation light ina state of being activated (hereinafter, referred to as activatedstate). The fluorescent dye may contain an activator dye that brings thereporter dye into the activated state upon receiving activating light.When the fluorescent dye does not contain an activator dye, the reporterdye is brought into the activated state upon receiving the activationlight. Examples of the fluorescent dye include a dye pair in which twotypes of cyanine dyes are bound (such as Cy3-Cy5 dye pair (Cy3, Cy5 areregistered trademarks), Cy2-Cy5 dye pair (Cy2, Cy5 are registeredtrademarks), and Cy3-Alexa Fluor 647 dye pair (Cy3, Alexa Fluor areregistered trademarks)), and a type of dye (such as, Alexa Fluor 647(Alexa Fluor is a registered trademark)). Examples of the fluorescentprotein include PA-GFP and Dronpa.

The microscope main body 51 includes a stage 102, a light source device103, an illumination optical system 104, a first observation opticalsystem 105, and an image capturer 106. The stage 102 holds a sample W tobe observed. The stage 102 can, for example, have the sample placed onan upper surface thereof. The stage 102 may have, for example, amechanism for moving the sample W as seen with an XY stage or may nothave a mechanism for moving the sample W as seen with a desk or thelike. The microscope main body 51 need not include the stage 102.

The light source device 103 includes an activation light source 110 a,an excitation light source 110 b, a shutter 111 a, and a shutter 111 b.The activation light source 110 a emits an activation light L whichactivates a part of the fluorescent substance contained in the sample W.Here, the fluorescent substance contains a reporter dye and contains noactivator dye. The reporter dye of the fluorescent substance is broughtinto the activated state capable of emitting fluorescence by irradiatingthe activation light L thereon. The fluorescent substance may contain areporter dye and an activator dye, and in such a case the activator dyeactivates the reporter dye upon receiving the activation light L. Thefluorescent substance may be a fluorescent protein such as PA-GFP orDronpa.

The excitation light source 110 b emits an excitation light L1 whichexcites at least a part of the activated fluorescent substance in thesample W. The fluorescent substance emits fluorescence or is inactivatedwhen the excitation light L1 is irradiated thereon in the activatedstate. When the fluorescent substance is irradiated with the activationlight L in the inactive state (hereunder, referred to as inactivatedstate), the fluorescent substance is activated again.

The activation light source 110 a and the excitation light source 110 binclude, for example, a solid-state light source such as a laser lightsource, and each emit a laser light of a wavelength corresponding to thetype of fluorescent substance. The emission wavelength of the activationlight source 110 a and the emission wavelength of the excitation lightsource 110 b are selected, for example, from approximately 405 nm,approximately 457 nm, approximately 488 nm, approximately 532 nm,approximately 561 nm, approximately 640 nm, and approximately 647 nm.Here, it is assumed that the emission wavelength of the activation lightsource 110 a is approximately 405 nm and the emission wavelength of theexcitation light source 110 b is a wavelength selected fromapproximately 488 nm, approximately 561 nm, and approximately 647 nm.

The shutter 111 a is controlled by the controller 53 and is capable ofswitching between a state of allowing the activation light L from theactivation light source 110 a to pass therethrough and a state ofblocking the activation light L. The shutter 111 b is controlled by thecontroller 53 and is capable of switching between a state of allowingthe excitation light L1 from the excitation light source 110 b to passtherethrough and a state of blocking the excitation light L1.

The light source device 103 includes a mirror 112, a dichroic mirror113, an acousto-optic element 114, and a lens 115. The mirror 112 isprovided, for example, on an emission side of the excitation lightsource 110 b. The excitation light L1 from the excitation light source110 b is reflected on the mirror 112 and enters the dichroic mirror 113.The dichroic mirror 113 is provided, for example, on an emission side ofthe activation light source 110 a. The dichroic mirror 113 has acharacteristic of transmitting the activation light L therethrough andreflecting the excitation light L1 thereon. The activation light Ltransmitted through the dichroic mirror 113 and the excitation light L1reflected on the dichroic mirror 113 enter the acousto-optic element 114through the same optical path.

The acousto-optic element 114 is, for example, an acousto-optic filter.The acousto-optic element 114 is controlled by the controller 53 and canadjust the light intensity of the activation light L and the lightintensity of the excitation light L1, respectively. The acousto-opticelement 114 is controlled by the controller 53 and is capable ofswitching between a state of allowing both the activation light L andthe excitation light L1 to pass therethrough (hereunder, referred to aslight-transmitting state) and a state of blocking or reducing theintensity of the activation light L and the excitation light L1(hereunder, referred to as light-blocking state). For example, when thefluorescent substance contains a reporter dye and contains no activatordye, the controller 53 controls the acousto-optic element 114 so thatthe activation light L and the excitation light L1 are simultaneouslyirradiated. When the fluorescent substance contains the reporter dye andcontains no activator dye, the controller 53 controls the acousto-opticelement 114 so that the excitation light L1 is irradiated after theirradiation of the activation light L, for example. The lens 115 is, forexample, a coupler, and focuses the activation light L and theexcitation light L1 from the acousto-optic element 114 on a light guide116.

The microscope main body 51 need not include at least a part of thelight source device 103. For example, the light source device 103 may beunitized and may be provided exchangeably (in an attachable anddetachable manner) on the microscope main body 51. For example, thelight source device 103 may be attached to the microscope main body 51at the time of performing an observation by means of the microscope 1.

The illumination optical system 104 irradiates the activation light L,which activates a part of the fluorescent substance contained in thesample W, and the excitation light L1, which excites at least a part ofthe activated fluorescent substance. The illumination optical system 104irradiates the sample W with the activation light L and the excitationlight L1 from the light source device 103. The illumination opticalsystem 104 includes the light guide 116, a lens 117, a lens 118, afilter 119, a dichroic mirror 120, and an objective lens 121.

The light guide 116 is, for example, an optical fiber, and guides theactivation light L and the excitation light L1 to the lens 117. In FIG.1 and so forth, the optical path from the emission end of the lightguide 116 to the sample W is shown with a dotted line. The lens 117 is,for example, a collimator, and converts the activation light L and theexcitation light L1 into parallel lights. The lens 118 focuses, forexample, the activation light L and the excitation light L1 on a pupilplane of the objective lens 121. The filter 119 has a characteristic,for example, of transmitting the activation light L and the excitationlight L1 and blocking at least a part of lights of other wavelengths.The dichroic mirror 120 has a characteristic of reflecting theactivation light L and the excitation light L1 thereon and transmittinglight of a predetermined wavelength (for example, fluorescence) amongthe light from the sample W. The light from the filter 119 is reflectedon the dichroic mirror 120 and enters the objective lens 121. The sampleW is placed on a front side focal plane of the objective lens 121 at thetime of observation.

The activation light L and the excitation light L1 are irradiated ontothe sample W by means of the illumination optical system 104 asdescribed above. The illumination optical system 104 mentioned above isan example, and changes may be made thereto where appropriate. Forexample, a part of the illumination optical system 104 mentioned abovemay be omitted. The illumination optical system 104 may include at leasta part of the light source device 103. Moreover, the illuminationoptical system 104 may also include an aperture diaphragm, anillumination field diaphragm, and so forth.

The first observation optical system 105 forms an image of light fromthe sample W. Here, the first observation optical system 105 forms animage of fluorescence from the fluorescent substance contained in thesample W. The first observation optical system 105 includes theobjective lens 121, the dichroic mirror 120, a filter 124, a lens 125,an optical path switcher 126, a lens 127, and a lens 128. The firstobservation optical system 105 shares the objective lens 121 and thedichroic mirror 120 with the illumination optical system 104. In FIG. 1and so forth, the optical path between the sample W and the imagecapturer 106 is shown with a solid line.

The fluorescence from the sample W travels through the objective lens121 and the dichroic mirror 120 and enters the filter 124. The filter124 has a characteristic of selectively allowing light of apredetermined wavelength among the light from the sample W to passtherethrough. The filter 124 blocks, for example, illumination light,external light, stray light and the like reflected on the sample W. Thefilter 124 is unitized with, for example, the filter 119 and thedichroic mirror 120 as a filter unit 23, and the filter unit 23 isprovided interchangeably. For example, the filter unit 23 may beexchanged according to the wavelength of the light emitted from thelight source device 103 (for example, the wavelength of the activationlight L, the wavelength of the excitation light L1), and the wavelengthof the fluorescence emitted from the sample W. Alternatively, a singlefilter unit corresponding to a plurality of excitation and fluorescencewavelengths may be used.

The light having passed through the filter 124 enters the optical pathswitcher 126 via the lens 125. The light leaving the lens 125 forms anintermediate image on an intermediate image plane 105 b after havingpassed through the optical path switcher 126. The optical path switcher126 is, for example, a prism, and is provided in a manner that allows itto be inserted into and retracted from the optical path of the firstobservation optical system 105. The optical path switcher 126 isinserted into or retracted from the optical path of the firstobservation optical system 105 by means of a driver (not shown in thedrawings) which is controlled by the controller 53. The optical pathswitcher 126 guides the fluorescence from the sample W to the opticalpath toward the image capturer 106 by means of internal reflection, in astate of having been inserted into the optical path of the firstobservation optical system 105.

The lens 127 converts the fluorescence leaving from the intermediateimage (the fluorescence having passed through the intermediate imageplane 105 b) into parallel light, and the lens 128 focuses the lighthaving passed through the lens 127. The first observation optical system105 includes an astigmatic optical system (for example, a cylindricallens 129). The cylindrical lens 129 acts at least on a part of thefluorescence from the sample W to generate astigmatism for at least apart of the fluorescence. That is to say, the astigmatic optical systemsuch as the cylindrical lens 129 generates astigmatism with respect atleast to a part of the fluorescence to generate an astigmaticdifference. This astigmatism is used, for example, to calculate theposition of the fluorescent substance in a depth direction of the sampleW (an optical axis direction of the objective lens 121).

The cylindrical lens 129 is provided in a manner that allows it to beinserted into or retracted from the optical path between the sample Wand the image capturer 106 (for example, an image-capturing element140). For example, the cylindrical lens 129 can be inserted into orretracted from the optical path between the lens 127 and the lens 128.The cylindrical lens 129 is arranged in this optical path in the modefor generating a three-dimensional super-resolution image, and isretracted from this optical path in the mode for generating atwo-dimensional super-resolution image.

In the present embodiment, the microscope main body 51 includes a secondobservation optical system 130. The second observation optical system130 is used to set an observation range and so forth. The secondobservation optical system. 130 includes, in an order toward a viewpoint Vp of the observer from the sample W, the objective lens 121, thedichroic mirror 120, the filter 124, the lens 125, a mirror 131, a lens132, a mirror 133, a lens 134, a lens 135, a mirror 136, and a lens 137.

The second observation optical system 130 shares the configuration fromthe objective lens 121 to the lens 125 with the first observationoptical system 105. After having passed through the lens 125, thefluorescence from the sample W enters the mirror 131 in a state wherethe optical path switcher 126 is retracted from the optical path of thefirst observation optical system 105. The light reflected on the mirror131 enters the mirror 133 via the lens 132, and after having beenreflected on the mirror 133, the light enters the mirror 136 via thelens 134 and the lens 135. The light reflected on the mirror 136 entersthe view point Vp via the lens 137. The second observation opticalsystem 130 forms an intermediate image of the sample W in the opticalpath between the lens 135 and the lens 137 for example. The lens 137 is,for example, an eyepiece lens, and the observer can set an observationrange by observing the intermediate image therethrough.

The image capturer 106 image-captures an image formed by the firstobservation optical system 105. The image capturer 106 includes animage-capturing element 140 and a controller 141. The image-capturingelement 140 is, for example, a CMOS image sensor, but may also be a CCDimage sensor or the like. The image-capturing element 140 has, forexample, a plurality of two-dimensionally arranged pixels, and is of astructure in which a photoelectric conversion element such as photodiodeis arranged in each of the pixels. For example, the image-capturingelement 140 reads out the electrical charges accumulated in thephotoelectric conversion element by means of a readout circuit. Theimage-capturing element 140 converts the read electrical charges intodigital data, and outputs digital format data in which the pixelpositions and the gradation values are associated with each other (forexample, image data). The controller 141 causes the image-capturingelement 140 to operate on the basis of a control signal input from thecontroller 53, and outputs data of the captured image to the controldevice 52. Also, the controller 141 outputs to the control device 52 anelectrical charge accumulation period and an electrical charge readoutperiod.

On the basis of a signal (image capturing timing information) indicatingthe electrical charge accumulation period and the electrical chargereadout period supplied from the controller 141, the controller 53supplies to the acousto-optic element 114 a control signal for switchingbetween the light-transmitting state where the light from the lightsource device 103 is allowed to pass through and the light-blockingstate where the light from the light source device 103 is blocked. Theacousto-optic element 114 switches between the light-transmitting stateand the light-blocking state on the basis of this control signal. Thecontroller 53 controls the acousto-optic element 114 to control theperiod during which the sample W is irradiated with the activation lightL and the period during which the sample W is not irradiated with theactivation light L. The controller 53 controls the acousto-optic element114 to control the period during which the sample W is irradiated withthe excitation light L1 and the period during which the sample W is notirradiated with the excitation light L1. The controller 53 controls theacousto-optic element 114 to control the light intensity of theactivation light L and the light intensity of the excitation light L1which are irradiated onto the sample W.

In place of the controller 53, on the basis of a signal (image capturingtiming information) indicating the electrical charge accumulation periodand the electrical charge readout period, the controller 141 may supplyto the acousto-optic element 114 a control signal for switching betweenthe light-transmitting state and the light-blocking state to therebycontrol the acousto-optic element 114.

The controller 53 controls the image capturer 106 to cause theimage-capturing element 140 to execute image capturing. The controldevice 52 acquires an image-capturing result (captured image data) fromthe image capturer 106. The image processor 54 calculates positioninformation of the fluorescent substance in each fluorescent image bycalculating the centroid of the fluorescent image in the captured image,and uses the calculated position information to generate point clouddata DG. In the case of a two-dimensional STORM, the image processor 54calculates two-dimensional position information of a fluorescentsubstance and generates point cloud data DG including a plurality ofpieces of two-dimensional data. In the case of a three-dimensionalSTORM, the image processor 54 calculates three-dimensional positioninformation of a fluorescent substance and generates point cloud data DGincluding a plurality of pieces of three-dimensional data.

The image processor 54 outputs the point cloud data DG to theinformation processing device 1 shown in FIG. 24. The informationprocessing device 1 processes the point cloud data DG obtained from thedetection result of the microscope main body 51. The control device 52may acquire the image capturing result (captured image data) from theimage capturer 106 and may output the acquired image capturing result tothe information processing device 1, and the information processingdevice 1 may generate the point cloud data DG. In such a case, asdescribed above, the image generator 13 of the information processingdevice 1 calculates the position information of the fluorescentsubstance of each fluorescent image, and generates the point cloud dataDG using a plurality of pieces of the calculated position information.The image generator 13 generates the point cloud image Im1 representingthe point cloud data DG. In the case of a two-dimensional STORM, theimage generator 13 calculates two-dimensional position information of afluorescent substance and generates point cloud data DG including aplurality of pieces of two-dimensional data. In the case of athree-dimensional STORM, the image generator 13 calculatesthree-dimensional position information of a fluorescent substance andgenerates point cloud data DG including a plurality of pieces ofthree-dimensional data.

An observation method according to the present embodiment includes:irradiating an activation light for activating a part of a fluorescentsubstance contained in a sample; irradiating an excitation light toexcite at least a part of the activated fluorescent substance;image-capturing an image of light from a sample; calculating positioninformation of the fluorescent substance, on the basis of an imagecapturing result obtained as a result of image capturing; generatingpoint cloud data, using the calculated position information; andprocessing the generated point cloud data by means of the informationprocessing method according to the embodiment. For example, the controldevice 52 controls the microscope main body 51, and thereby, themicroscope main body 51 detects an image of fluorescence emitted from asample containing a fluorescent substance. The control device 52controls the microscope main body 51 to generate point cloud data. Thecontrol device 52 controls the information processing device 1 to causethe field calculator 7A to calculate a function representing a scalarfield. The control device 52 controls the information processing device1 to cause the UI 6 to acquire user input information including theinformation which specifies the scalar value in the scalar field. Thecontrol device 52 controls the information processing device 1 to causethe surface generator 7B to generate surface information representingthe isosurface of the scalar value corresponding to the inputinformation.

In the embodiment described above, the control device 52 includes, forexample, a computer system. The control device 52 reads out anobservation program stored in the memory storage and executes variousprocesses according to the observation program. This observation programcauses a computer to execute: control of irradiating an activation lightfor activating a part of a fluorescent substance contained in a sample;control of irradiating an excitation light to excite at least a part ofthe activated fluorescent substance; control of image-capturing an imageof light from a sample; calculating position information of thefluorescent substance, on the basis of an image capturing resultobtained as a result of image capturing; generating point cloud data,using the calculated position information; and processing the generatedpoint cloud data. This observation program may be recorded and providedon a computer-readable storage medium (for example, a non-transitoryrecording medium, or a non-transitory tangible medium).

The technical scope of the present invention is not limited to the modesdescribed in the above embodiment and so forth. One or more of therequirements described in the above embodiments and so forth may beomitted. One or more of the requirements described in the aboveembodiments and so forth may also be combined where appropriate.Furthermore, the contents of all documents cited in the detaileddescription of the present invention are incorporated herein byreference to the extent permitted by law.

DESCRIPTION OF REFERENCE SIGNS

-   1 Information processing device-   2 Display device (display)-   3 Input device-   6 UI-   7 Field calculator-   8 Surface generator-   9 Memory storage-   15 Processor-   50 Microscope-   51 Microscope main body-   52 Controller

What is claimed is:
 1. An information processing device comprising: animage processor which generates surface information from point clouddata generated on the basis of position information of a fluorescentsubstance contained in a sample, using a value of a first parameter ofthe surface information and a value of a second parameter of the surfaceinformation; and a display controller which causes a display to displaya surface image on the basis of the generated surface information,wherein the image processor generates a plurality of pieces of thesurface information of a plurality of combinations, respectively, of aplurality of values of the first parameter and a plurality of values ofthe second parameter, and the display controller causes the display todisplay a plurality of surface images on the basis of the plurality ofpieces of surface information, respectively, of the plurality ofcombinations.
 2. The information processing device according to claim 1,wherein the display controller causes the display to display theplurality of surface images in an arranged manner.
 3. The informationprocessing device according to claim 1, wherein the display controllercauses the display to display the surface image at a positioncorresponding to the value of each parameter where the vertical axisrepresents the values of the first parameter and the horizontal axisrepresents the values of the second parameter.
 4. The informationprocessing device according to claim 1, wherein the display controllercauses the display to display the plurality of surface images such thatthe plurality of values of the first parameter correspond to a rowdirection and such that the plurality of values of the second parametercorrespond to a column direction.
 5. The information processing deviceaccording to claim 1, wherein the display controller causes the displayto display the plurality of surface images such that the plurality ofvalues of the first parameter correspond to a vertical axis directionand such that the plurality of values of the second parameter correspondto a horizontal axis direction.
 6. The information processing deviceaccording to claim 1, wherein the display controller causes the displayto display a setting screen to set the value of the first parameter andthe value of the second parameter, and displays, on the setting screen,the value of the first parameter and the value of the second parametercorresponding to a surface image selected from the plurality of surfaceimages displayed on the display.
 7. The information processing deviceaccording to claim 1, wherein the image processor calculates a scalarfield, applying a predetermined function in a region within apredetermined distance from each point data of the point cloud data,discretizes a scalar field at a grid point, and calculates the surfaceinformation, calculating an isosurface of a scalar value of the gridpoint, and the value of the first parameter and the value of the secondparameter are any one of a value related to the size of the region, avalue related to the resolution of the grid point, and a value of theisosurface.
 8. A microscope comprising: the information processingdevice according to claim 1; an illumination optical system whichirradiates an excitation light to excite the fluorescent substancecontained in the sample; an observation optical system which forms animage of light from the sample; and an image capturer which captures theimage formed by the observation optical system, wherein the imageprocessor calculates the position information of the fluorescentsubstance on the basis of a result captured by the image capturer, andgenerates the point cloud data, using the calculated positioninformation.
 9. A microscope comprising: the information processingdevice according to claim 1; an optical system which irradiatesactivation light for activating a part of the fluorescent substancecontained in the sample; an illumination optical system which irradiatesan excitation light to excite at least a part of the activatedfluorescent substance; an observation optical system which forms animage of light from the sample; and an image capturer which captures theimage formed by the observation optical system, wherein the imageprocessor calculates the position information of the fluorescentsubstance on the basis of a result captured by the image capturer, andgenerates the point cloud data, using the calculated positioninformation.
 10. An information processing method for generating surfaceinformation from point cloud data generated on the basis of positioninformation of a fluorescent substance contained in a sample, using avalue of a first parameter of the surface information and a value of asecond parameter of the surface information, and causing a display todisplay a surface image on the basis of the generated surfaceinformation, the information processing method comprising: generating aplurality of pieces of the surface information of a plurality ofcombinations, respectively, of a plurality of values of the firstparameter and a plurality of values of the second parameter; and causingthe display to display a plurality of surface images on the basis of theplurality of pieces of surface information, respectively, of theplurality of combinations.
 11. A non-transitory computer-readabletangible medium containing an information processing program whichcauses a computer to execute processes of: generating surfaceinformation from point cloud data generated on the basis of positioninformation of a fluorescent substance contained in a sample, using avalue of a first parameter of the surface information and a value of asecond parameter of the surface information, and causing a display todisplay a surface image on the basis of the generated surfaceinformation, the information processing program causing the computer toexecute: generating a plurality of pieces of the surface information ofa plurality of combinations, respectively, of a plurality of values ofthe first parameter and a plurality of values of the second parameter;and causing the display to display a plurality of surface images on thebasis of the plurality of pieces of surface information, respectively,of the plurality of combinations.